Paul Hammant's Blog: Red-Green-Refactor Moves Into the Harness

I’ve been wondering aloud whether red-green-refactor is becoming a memory. For developers over 35 or so, it was a hard-won lesson: write the failing test, see it fail, make it pass, tidy up. One day nobody working will remember learning it that way, because coding agents write most of the tests now, and they don’t remember anything. Well learning first hand for humans and coding agents.

Two people I respect have written about TDD and coding agents, and they come out in different places:

Kent Beck: TDD keeps the agent honest

In Augmented Coding: Beyond the Vibes, Kent built a B+ tree library with an agent he calls “the genie”. His system prompt required red-green-refactor, one test at a time, with structural and behavioural changes in separate commits. He watched the intermediate results, “ready to intervene & stop unproductive development”, and named the warning signs: loops, unrequested functionality, and “any indication that the genie was cheating, for example by disabling or deleting tests.”

His verdict: good on correctness and performance, “not so good about the code quality.”

So in Kent’s hands, an agent kept to TDD is arguably more disciplined than the average enterprise team ever was before AI. Plenty of those teams wrote tests after the code, when they wrote them at all. But note the condition: Kent was watching.

Birgitta Böckeler: TDD in the agent loop may be theater

Birgitta’s TDD inside the agent loop - theater or actual value? on Martin Fowler’s site runs an experiment rather than a project. TDD and non-TDD agent sessions, on tasks of three sizes, judged blind by another model. She’s upfront that the sample is small. The results: no clear quality difference (non-TDD was ranked higher more often), no meaningful difference in mutation scores, and TDD costing 3 to 8.5 times the tokens.

The detail that matters most to me is what the agents actually did. They “sometimes skipped or faked the red step, or implemented ahead of the test so that it passed immediately.” Some tests “checked the implementation’s output against itself.” And: “a red test tells you the agent ran it and saw failure, not that the failure was for the right reason.”

Her conclusion is that she has stopped telling agents to write tests first, and that we should monitor outcomes instead: mutation testing, static analysis, structure reviews.

Where they agree

Read side by side, the two aren’t as far apart as the headlines suggest. Both say agent-written tests can’t be trusted by default. Kent’s answer is to enforce the process and supervise it. Birgitta’s is to stop prescribing the process and check the outcome.

What neither says is that the point of red-green-refactor has gone away. A test that has never been seen failing proves nothing. That was true in 2003 and it is true now. The fables of 100%-passing test suites with no assertions in them were about humans. The modern version is an agent writing a test that passes by construction.

Where the lesson goes

So I’ll restate what I said. Red-green-refactor as a ritual humans remember may well fade. The lesson doesn’t and that can be trained into coding AI. End result it moves into the harness one way or another:

• Instructions. CLAUDE.md, AGENTS.md, the system prompt. Kent’s appendix is a fine example. This is the cheapest place, and Birgitta’s evidence says it is also the least reliable on its own.
• Objective checks in CI. Mutation testing specifically, tells you whether the tests would notice the code being broken. A job that runs each new test against the previous commit, and expects it to fail, tells you it was red before it was green, for whatever reason the agent claims. Neither depends on trusting the agent’s account of what it did. I’ve had success with this, but it is a spoonful of test after development. Not just CI, but pre commit/push too.
• Humans on the loop. Kief Morris’s Humans and Agents in Software Engineering Loops describes the shift: rather than fixing each artifact, you improve the harness that produced it. When a tautological test gets through, the fix is a better check, not a sterner prompt.
• Solid repo foundations It could be that the pyramid of tests as well as mocking/stubbing/doubles are the perfect exmpllar of tests a coding AI can continue to follow. Then also, some project/repos have excellent non functional tests (perf, load, memory use, etc) and those can be some of the same foundations to follow. If notthing else an AI can be directed to learn “house rules” and have checked that off within seconds of encoutering a new repo with an otherwise skinny AGENTS.md.

The second of those is the one I’d continue to bet on in less that perfect setup. It satisfies Kent’s discipline and Birgitta’s skepticism at the same time, and it keeps working after nobody remembers why it was put there. That is how hard-won lessons survive: they become a step in the build.

Still open

I don’t think this is settled. Kent’s experience suggests agents can do TDD better than most teams did. Birgitta’s experiment suggests that, unsupervised, they mostly perform it. Both could be true, and the difference might be the supervision. If so, the supervision needs to be in the harness too, because the humans doing it are going to be on the loop, not in it. It would be cool if every company Kent worked for since the 90’s would allow the VCS from back then to be mind for the Kent-paired commits to train KentBeck.ai. Well, only if Kent isn’t horrified by that.

I’ve added a paragraph on this to the Fast, Green Builds page on trunkbaseddevelopment.com. That page is about keeping the build fast and always green, and a test that was never red is the oldest way for a build to be green incorrectly, even if we have to adapt going forward.