cats: REST API Fuzzer and negative testing tool

by do son · August 1, 2022

cats

REST API fuzzer and negative testing tool. Run thousands of self-healing API tests within minutes with no coding effort!

Comprehensive: tests are generated automatically based on a large number of scenarios and cover every field and header
Intelligent: tests are generated based on data types and constraints; each Fuzzer has specific expectations depending on the scenario under the test
Highly Configurable: high amount of customization: you can exclude specific Fuzzers, HTTP response codes, provide business context, and a lot more
Self-Healing: as tests are generated, any OpenAPI spec change is picked up automatically
Simple to Learn: flat learning curve, with intuitive configuration and syntax
Fast: automatic process for write, run and report tests which covers thousands of scenarios within minutes

By using a simple and minimal syntax, with a flat learning curve, CATS (Contract Auto-generated Tests for Swagger) enables you to generate thousands of API tests within minutes with no coding effort. All tests are generated, run, and reported automatically based on a pre-defined set of 89 Fuzzers. The Fuzzers cover a wide range of input data from fully random large Unicode values to well crafted, context dependant values based on the request data types and constraints. Even more, you can leverage the fact that CATS generates request payloads dynamically and write simple end-to-end functional tests.

Understanding how CATS works and reports results

CATS generates tests based on configured Fuzzers. Each Fuzzer has a specific scenario and a specific expected result. The CATS engine will run the scenario, get the result from the service and match it with the Fuzzer expected result. Depending on the matching outcome, CATS will report as follows:

INFO/SUCCESS is expected and documented behaviour. No need for action.
WARN is expected but undocumented behaviour or some misalignment between the contract and the service. This will ideally be actioned.
ERROR is abnormal/unexpected behaviour. This must be actioned.

CATS will iterate through all endpoints, all HTTP methods, and all the associated requests bodies and parameters (including multiple combinations when dealing with oneOf/anyOf elements) and fuzz their values considering their defined data type and constraints. The actual fuzzing depends on the specific Fuzzer executed. Please see the list of fuzzers and their behaviour. There are also differences in how the fuzzing works depending on the HTTP method:

for methods with request bodies like POST, PUT the fuzzing will be applied at the request body data models level
for methods without request bodies like GET, DELETE the fuzzing will be applied at the URL parameters level

This means that for methods with request bodies (POST,PUT) that have also URL/path parameters, you need to supply the path parameters via urlParams or the referenceData file as failure to do so will result in Illegal character in path at index … errors.