Agent Verification Loop

by Ray Svitla

agents declare success the same way they declare failure — confidently, in fluent prose. the verification loop runs between “done” and a write actually landing in your repo, inbox, CDN, or database.

seven checks. cheap to run, expensive to skip. run them before any agent action that writes, sends, deploys, deletes, or publishes. coding agents and publishing agents fail in the same shape — only the artifacts differ.

when to run it

run the loop when the next agent action is one of these:

skip it for read-only thinking, search, or planning. the loop is overhead. spend it where reversal is hard.

the seven checks

each check is one question the operator (you, or another agent acting as verifier) has to answer “yes” to. one “no” stops the loop.

1. intent — did the agent restate what was actually asked?

before any action, the agent re-states the goal in its own words. compare to the original request. mismatches surface here, not after a database row goes missing.

user: "deploy the staging build"
agent: "i'm about to deploy commit abc123 to production."

stop. that’s a goal misinterpretation, not a deploy command. classic agent failure mode — the wrong-objective family.

operator checklist:

• does the restated goal match the request on scope (which env, which file, which target)?
• did the agent invent any constraints not in the request?
• did the agent drop any constraints from the request?

2. artifact — does the named thing actually exist?

agents hallucinate file paths, package names, command flags, function names, branch names, env vars, slugs. the artifact check is one shell call (or one read) confirming the named thing exists before you act on it.

if the artifact does not exist, the agent does not invent it. it stops and asks.

3. command/test — did the command actually run, with output?

the agent ran the command, saw the output, and the output is in the transcript. not a summary. not “tests pass.” the actual stdout.

minimum bar:

• the agent ran the test/lint/typecheck/build command (not just claimed it)
• the output is readable in the transcript
• the exit code is visible
• failures are quoted, not paraphrased

a paraphrase like “the test suite passed” without output is a verification failure wearing a confident voice. the model can hallucinate the green check just as easily as it hallucinates a file path.

4. diff — did a human look at the patch?

before commit, before push, before publish, read the diff. not the summary the agent generated. the diff itself.

git --no-pager diff --stat
git --no-pager diff

read it as a person, not as a model. things to catch:

• files you didn’t expect changed
• secrets, tokens, credentials, env values
• comments saying “removed” or “temporarily disabled” that hide deletions
• whitespace-only “fixes” hiding logic changes
• generated files committed by mistake
• stray dependency bumps in package.json / requirements.txt

for publishing agents, the equivalent is: open the rendered output. read it as the reader will see it. don’t trust the agent’s plaintext summary of what got published.

5. source — where did the claim come from?

every non-obvious factual claim in agent output should have a source you can re-fetch. for each claim, the operator can answer one of:

• the agent quoted the source and the URL is reachable
• i ran the command myself and saw the same value
• the claim is local and verifiable (git log, file contents, build output)

if none of those, it’s a hallucination cascade waiting to compound. agent says “the latest version is X,” ships an upgrade against a number it invented, and the dependency surface is wrong for a week.

for publishing agents: every quote, every star count, every benchmark number, every “according to” — has to have a primary source. community chatter is signal, not citation.

6. rollback — what’s the undo, and how long does it take?

before action, write the rollback. one line. concrete.

if the rollback is “ask the platform team,” the loop stops. you do not run an action whose undo lives in someone else’s calendar.

7. stop — when does the loop end?

agents loop forever unless you tell them not to. explicit stop:

• the original goal is satisfied (intent restated and matched at the end)
• all artifacts named in the plan exist
• all commands ran with visible output and zero non-acceptable failures
• the diff or rendered artifact has been read by a human or a verifying agent that did not write it
• sources for non-obvious claims are present and reachable
• rollback is documented and runnable

if any check fails, the agent does not retry silently. it surfaces the failure with the failed check named.

if all checks pass, the agent stops. no extra “improvements.” no scope drift. no “while i’m here.”

a real run — claim, evidence, decision

task: “add a feature flag for the new export route.”

claim      "EXPORTS_V2 added; tests pass; safe to commit."

evidence   $ ls src/routes/exports.ts
           src/routes/exports.ts

           $ grep -rn EXPORTS_V2 src/
           src/routes/exports.ts:12: if (process.env.EXPORTS_V2 === "true") {

           $ pnpm typecheck
           ...
           no errors. exit 0.

           $ pnpm test routes/exports
           PASS  src/routes/exports.test.ts
             ✓ flag off → 404
             ✓ flag on  → 200 + payload
           Tests: 2 passed.    Exit code: 0.

           $ git --no-pager diff --stat
            src/routes/exports.ts | 18 ++++++++++++++-----
            1 file changed, 14 insertions(+), 4 deletions(-)

           source  docs/feature-flags.md (in-tree pattern)

decision   proceed to commit on feature branch.
           rollback: `git revert <sha>`; flag default off keeps prod
           safe even on accidental merge.

every step is replayable. the agent has nowhere to hide a paraphrase.

for a publishing agent the shape is identical — ls becomes slug free, pnpm test becomes hugo --buildDrafts=false && htmltest, the diff is the rendered page in a browser, and the rollback is the unpublish path plus a CDN purge.

an installable shape — .verify.json

the loop is a workflow until you wire it into the harness. one cheap pattern: have the agent write a small .verify.json next to the action it’s about to take, then have a hook refuse the action unless the file is present and well-formed.

{
  "intent": "add EXPORTS_V2 flag gating /exports, off by default",
  "artifacts": [
    "src/routes/exports.ts",
    "src/routes/exports.test.ts"
  ],
  "commands": [
    { "cmd": "pnpm typecheck",           "exit": 0 },
    { "cmd": "pnpm test routes/exports", "exit": 0 }
  ],
  "diff_reviewed_by": "human",
  "sources": ["docs/feature-flags.md"],
  "rollback": "git revert <sha>; flag default off",
  "stop": "all seven checks green; no scope drift"
}

the rules:

• missing field → refuse.
• a command listed but no recorded exit code in the transcript → refuse.
• diff_reviewed_by empty → refuse.
• rollback empty or “ask someone” → refuse.

no policy engine, no DSL, just a guard with named slots. the file is the contract; the hook is the enforcement.

claude code hooks are the right place for this. the names of the events matter less than the shape: a hook script that reads .verify.json before any write/send/deploy/delete/publish tool call and refuses anything missing.

skills carry the same loop: a verify skill that activates on those keywords moves the loop from “discipline” to “default.” the rules belong in a file the agent loads, not in the human’s head.

coding agent vs publishing agent

the seven checks are the same. the artifacts differ.

the publishing failure mode that maps to “rm -rf the wrong dir” is “publish a confident piece with a fabricated quote.” both ship. both are hard to take back. both feel like victory at the moment of execution.

what this loop catches

it does not catch prompt injection or memory corruption — those need separate guardrails. it does not stop a determined operator from skipping it. it does stop the most common form of fake success: a confident “done” with nothing underneath.

it operationalizes the reflection pattern for actions, not just outputs. reflection self-critiques what the agent wrote. verification asks whether the world matches what the agent said.

related

• agent failure modes — the catalog this loop is defending against
• agentic design patterns — reflection, planning, tool use
• agent config files compared — where to record “before any write, run verify”
• best mcp servers for claude code — tools the loop calls during checks