I spend a lot of time with acceptance criteria written for people: a paragraph, a ticket, a short brief in a chat. That used to be enough. A person would argue with the paragraph, ask what "reasonable" meant, and push back when the note contradicted the last release. An AI coding agent does not argue. It treats the paragraph as atmosphere and the repository as ground truth. If the ground truth is empty, it invents a shape that looks like every other endpoint on the internet.
That is not a failure of the model. It is a failure of what we handed it.
Prose is not an executable contract
A sentence like "return a useful error when the sku is missing" can mean four different status codes and three different bodies. A person will pick one and defend it in review. An agent will pick the median example from its training and move on. The ticket still looks satisfied. The suite still says nothing about the choice.
The only description an agent is forced to respect is the one that fails a run. expect(res.status).toBe(400) and expect(res.body.error).toBe("sku") leave no room for a polite 422 with a different key. The agent can still write bad code. It cannot call the job done while that assertion is red.
So the order of work has to change when an agent is in the loop. The criteria stop being a note for later and become the first files you commit.
Write the tests before the implementation
I do not mean a ceremony. I mean a short suite that states the contract in the same language the CI will run:
- Happy path: status, required fields, the one invariant the product cannot bend.
- The refusals that matter: empty input, unknown id, a caller without the right scope.
- One check against a neighbour: the new response does not break the shape an old client already parses.
Those cases can be unfinished in the sense that the production code is not there yet. They should not be unfinished in the sense that the expectations are vague. If you cannot write the assertion, you do not yet know what you are asking for. The agent will not know either. It will guess.
What the agent is allowed to touch
Once the suite is there, the agent's job is narrow: make the red cases green without rewriting the cases. That rule is the whole review. If the agent softens an assertion to match the code it preferred, it has edited the spec. Treat that the way you would treat a silent change to an OpenAPI field: reject the diff, put the expectation back, and ask for another attempt.
Green is necessary, not sufficient. An agent can satisfy three assertions by special-casing three inputs, and the suite will cheer. So the review still reads the code, and when a shortcut shows up, the fix is another test, not a longer prompt. Every hole the agent finds becomes a case it can no longer slip through.
The prose ticket can stay. I still write one. It is useful for people, for product, for the pull request description. It is not the document I trust the agent to obey. The suite is.
A generated suite from a formal spec is a different story, and I have written about that elsewhere. This is the smaller habit for day-to-day agent work: before you ask for the function, give the agent something that can fail. Then the acceptance criteria are not a wish. They are a run.