Engineering
Agentic API testing: beyond the 200 OK
Status codes tell you an endpoint answered. An agent can tell you whether it answered correctly, consistently and safely.
Most API monitoring answers one question: did the endpoint respond? A 200 OK is reassuring, but it says nothing about whether the body has the right shape, whether bad input is rejected, or whether an endpoint that should require a token actually does. Those are the questions our API tester is built to answer.
What a status code tells you
A status code is the API's own opinion of how the request went. It is useful, and it is also easy to get wrong. Endpoints return 200 with an error message in the body. They return 500 for what should be a validation error. They return 201 for a record that should never have been created.
A conventional test suite catches these if someone wrote a test for each case. In practice, most suites cover the happy path well and the edges thinly, because writing edge-case tests is tedious. An agent is good at tedious, systematic exploration, and it can explain what it found in plain language.
How a test run works
You describe the API; the agent designs and runs the tests.
- 01
Describe the endpoints
Give a base URL and a list of endpoints, each with a method and path. ForPOST,PUTandPATCHyou can include a JSON request schema; any endpoint can take query parameters. A single run covers up to 100 endpoints. - 02
Configure headers
Global headers, such as anAuthorizationbearer token or an API key, are applied to the run. Use credentials for a test account or staging environment. - 03
The agent generates test cases
For each endpoint it writes a set of named test cases with synthetic payloads: valid requests, invalid ones, missing fields, boundary values, and variations on authentication. - 04
Requests run against your API
Each case is sent for real, and the request headers, body, response status and response body are recorded. Progress streams to the results page while the run continues in the background. - 05
Results are assessed and scored
Every case is marked passed or failed against what a correct API should do. The run gets an overall score out of 100 with written reasoning.
What the agent checks
The point of using an agent is that it judges responses, not just collects them. These are the kinds of questions it asks of each endpoint.
Response shape
Does the body match what the endpoint is supposed to return? If the request schema implies a user object with an id and an email, a response that wraps it in an unexpected envelope, drops a field or returns a string where a number belongs is worth flagging, even with a 200.
Error handling
When a request is wrong, does the API say so clearly? Good APIs return a 4xx status with a useful message for client mistakes and reserve 5xx for real server faults. The agent looks for errors reported as successes, stack traces or internal details leaking into error bodies, and generic responses that would leave a client guessing.
Auth behaviour
Endpoints that should be protected are tried without credentials and with credentials that are malformed. An endpoint that returns data to an anonymous request, or answers a bad token with 500 instead of 401, is a finding.
Edge-case inputs
Empty strings, missing required fields, wrong types, very long values, unexpected characters and out-of-range numbers. This is where most bugs live, because it is where most hand-written test suites stop.
Consistency
Do similar endpoints behave alike? If one endpoint returns errors as { error: string } and another as { message: string }, or field names switch between camelCase and snake_case, every client has to handle both. The agent notes these patterns across the run.
An example finding
Here is a hypothetical case of the kind the tester produces. The endpoint answers with a success status, so a simple uptime check would pass it. The agent does not.
POST /users
Content-Type: application/json
Authorization: Bearer <test token>
{ "email": "not-an-email", "name": "" }
← 201 Created
{ "id": "u_1042", "email": "not-an-email", "name": "" }
FAILED · edge-case input
Expected a 4xx validation error for a malformed email and an
empty name. The endpoint accepted the payload and returned a
created record, so invalid data can reach storage. The valid-
payload case for this endpoint passed.The value is in the last paragraph. It states what was expected, what happened, why it matters, and how this case relates to the others for the same endpoint. That is the difference between a red cell in a dashboard and something a developer can act on.
Reading the results
Results are grouped by endpoint. Each endpoint is marked passed, partially failed or failed, and expands into its individual test cases. Every case shows the exact request that was sent and the response that came back, so you can reproduce it with any HTTP client.
At the top of the report is an overall score out of 100 with the assessor's reasoning, based on reliability, response validation and security behaviour across the run. From there you can share the report with a link, download it as a PDF, or re-run the same test after a fix to confirm it is resolved.
Getting useful results
- Point it at a test environment. The agent sends real requests, including writes and deletes. Use staging or a sandbox, not production data.
- Include request schemas. A schema for each write endpoint lets the agent build realistic valid payloads and targeted invalid ones.
- Use a dedicated test credential. Supply it as a global header so auth checks have a known-good baseline to compare against.
- Test related endpoints together. Consistency findings only appear when the agent can see several endpoints side by side.
A 200 OK means an endpoint is reachable. Whether it's correct is a separate question, and that one needs someone, or something, to actually read the response.