An AI agent's “Done” is a claim,
not a fact.

Measuring the thing next to the thing, and calling it the thing

False reports are born as verification failures, not as lies. The agent runs something that looks like verification and honestly reports what it saw. It just wasn't looking at the thing it claimed to be looking at. Running several small products for more than six months, I kept hitting the same four shapes.

Measuring one hop short

A curl without -L returns 301, and that becomes the evidence for “the page is serving correctly.” The body at the end of the redirect was never fetched.

Reading the wrong denominator

An analytics events count is reported as a visitor count. Nothing looks suspicious, because the number is genuine — it just answers a different question.

Calling an API 200 “it exists”

A success response becomes “the post is live.” “The request was accepted” and “the artifact exists on the other end” are two different facts.

Silent omission

“All rows clean” — after the parser skipped four lines mid-file without raising. The check ran correctly, against a population that was missing rows.

A rule that says “run a verification command” can make an agent run a check. It cannot make the check aim at the right thing. And as long as the verification layer lives inside the agent, verification itself is part of the work being optimized — and can be routed around.

What this harness does

It is designed to plug into Claude Code / Agent SDK hooks. The evidence it judges on is not what the agent submits — logs, traces, self-assessment — but external state.

• Measure the live HTTP response, the row in the database, what remains on the far side of an API, with fixed probes the agent cannot choose the shape of
• No model in the loop, no judge, no scoring: measure deterministically, then diff the claim against the measurement
• Accumulate the reconciliation as history, so that a per-agent, per-task rate of “claim disagreed with measurement” becomes visible

It isn't a guardrail that runs before or during the action to prevent something, and it isn't evals that score output quality offline. It's a layer that measures after the fact, from the outside, deterministically.

The honest boundary — if a 50-line hook covers you, use the 50-line hook

First, who shouldn't need this. If you're one person on one or two products and your outputs live inside the repo, a hand-rolled hook probably covers you. Free and small already handles all of this:

• Matching completion language against evidence that the work ran (a 50-line Stop hook does it)
• Prompt-level discipline that tells the agent to run verification commands
• Separating implementer from verifier using in-repo evidence: test results, diffs
• Deterministic checks in CI: lint, tests, secret scanning

Start there. I'm not trying to pull anyone in that range out of it.

What I found doesn't fit into the 50-line version: a maintained set of external-state probes (every trap above was learned by getting burned, and each one has to be baked in so it can't be re-chosen), the history of disagreement rates, a cross-repo view you can show a teammate or an auditor, and maintenance of the verification layer itself — the place that takes everything downstream with it when it breaks quietly. If you're stuck on the far side of that boundary, I'd like to hear about it.

Waitlist

An email address is all that's required. One optional question comes with it — what's the worst part for you right now. The distribution of those answers is what I'll build against, in that order.