the bottleneck moved to verification

The usual story about agentic coding is that it makes code cheap. That is true, but incomplete. The more immediate effect is that it moves scarcity somewhere less photogenic: into the systems that decide whether a change is worth trusting.

Anthropic has a concrete version of that problem. In a recent engineering account, the company says its CI job volume grew 25× in six months, while the test suite itself grew 10×. The service that worked out which tests each pull request needed began to buckle under that demand. Three capacity fixes bought 70 days, then 29, then less than a day. Eventually, Anthropic rebuilt the service around horizontally scaled test selection.

That story is not really about a clever CI architecture. It is about the order in which a team discovers its constraint. First, the models make it possible to open more changes. Then the test suite becomes the waiting room. Then the thing that decides which tests to run becomes production infrastructure.

Read Anthropic’s account of scaling test-impact analysis.

Speed makes a weaker promise than confidence

A generated patch can be plausible almost immediately. Confidence cannot.

Confidence needs a chain: a reason the change exists, an understanding of what it touches, tests chosen for the actual risk, someone able to see the resulting behaviour, and a way back when the judgement was wrong. None of that disappears when the first draft arrives in seconds. In some teams it gets harder, because the number of possible drafts rises faster than the team’s appetite for reviewing them.

This is why “we added an agent” is not a workflow description. The actual workflow begins when the agent has produced something that looks usable. Who owns the specification? What evidence counts as a pass? Which changes need a human read, and which can be tested in a narrower lane? Where does the decision get recorded so the next person can challenge it?

Without those answers, speed turns into a very efficient way of manufacturing uncertainty.

Test selection is editorial work

There is a temptation to treat testing as a binary gate: green means safe; red means stop. At scale, that gate has to make editorial decisions.

Running every test on every change is often too slow or too expensive. Running too few means a clean build can simply mean the relevant failure was never invited to appear. A test-impact system is therefore making an argument about relevance. It is saying: these files, these dependencies, these checks are the ones that matter for this patch.

That argument is not neutral. It contains a model of the codebase and an implied risk policy. As the codebase changes, the policy needs to be inspectable and revisable—not just fast.

Anthropic’s account is useful precisely because it shows this unglamorous layer under pressure. The first scaling patches were not durable. That is normal. What matters is that the selection system became visible as a system with its own failure modes, rather than remaining a hidden utility underneath the exciting part of AI-assisted programming.

Plans need to remain interruptible

The same applies upstream of CI. A plan produced by an agent may be coherent and still be wrong in the only way that matters: wrong after it meets the actual work.

Maggie Appleton makes the case through Lucy Suchman’s distinction between a plan and situated action. A plan can orient you, but it cannot do the navigating once wind, current, timing, and other people intervene. With agents, that is a design problem. The plan should not become a sealed artefact that the user is asked to approve and then quietly live beneath. The assumptions, constraints, and unresolved calls need to remain close to the work.

Read Maggie Appleton on planning with agents.

The good agent interface is not the one that makes deliberation invisible. It is the one that lets someone interrupt it without having to reconstruct the entire premise from a chat transcript.

Build the receipt, not just the output

For a small team, this does not require an enterprise process. It means keeping a compact receipt beside consequential work:

Claim: what is changing and why.
Boundary: what the change must not affect.
Evidence: the test, source, or visible behaviour that will decide whether it worked.
Owner: who can revise the call when reality disagrees.

That receipt is more useful than another dashboard full of agent activity. It makes the work legible while it is still cheap to correct.

Agentic coding will continue to reduce the cost of producing code. The operational advantage will go to teams that reduce the cost of knowing what to trust—and that leave enough of their reasoning exposed for the next interruption.

Reading

Personal AI OS tools — the control-plane map for personal agents, receipts, memory, and tools
Agent memory systems — what agents should remember, what belongs in logs, and how to avoid memory sludge

agent-workflows verification