Pre-send fact check

Agents sometimes state things with confidence that they never checked: a count they did not run, a status they assumed, "nothing like that exists" after a partial search.

Pre-send fact check adds a second reader. When a session finishes a turn with a real answer in it, a separate model reads the final reply next to the work the agent did in that turn (the files it read, the commands it ran and their output). If the reply states a number, a name, a status, or that something does not exist, and nothing the agent saw backs it up, the check flags it.

It also looks back through the session. When the reply repeats a specific number, name or identifier from earlier work, the few earlier results that mention it are included too, marked Earlier turn, so a wrong number repeated from an earlier turn can be caught. A bare number with nothing specific next to it ("4 views") usually cannot be matched this way.

The checker only compares. It never searches the web, runs anything or edits anything. Every flag names the claim and either quotes the evidence that contradicts it or says nothing in the turn covers it. A flag whose quote is not really in the evidence is thrown away before you see it.

Turn it on#

1. 1

   Open Settings > Sessions and find Pre-send fact check.

2. 2

   Turn on Check agent replies. It starts in Watch only, so nothing is sent to the agent while you see how accurate the flags are.

3. 3

   When you trust the flags, pick Send back.

The three modes#

• Off (the default): replies are not checked and nothing is spent.

• Watch only: the check runs and a small card under the reply says what would have been sent back. Nothing else changes.

• Send back: if any flag holds up, Yardmaster sends the agent one automated follow-up listing the claims and the one check to run for each. The agent runs the checks and gives the corrected reply in full. That corrected reply is checked once more, but it is never sent back a second time.

Only replies to a message you sent are checked. Turns an agent started on its own, scheduled messages, recipe and pipeline runs, and messages from other agents are left alone. Very short replies, and replies with nothing checkable in them, (plans, explanations, code) are skipped for free, without calling any model.

While a check runs, the session keeps showing as running: no call-back, no chime and no Inbox row until it finishes. The reply text still streams in as usual. A check that takes longer than 45 seconds is given up and the turn finishes normally. If you send another message during a check, it goes straight through and the check stops.

Which engines are covered#

• Claude Code, Codex and Pi: fully covered, checked against the turn's tool results, and Send back works.

• OpenClaw: watch only. OpenClaw does not report its tool results, so its replies are checked on their own and never sent back.

• Other engines (Gemini, opencode, Cursor and the rest), and sessions on a managed OpenRouter route, are not checked.

For Codex, Pi and OpenClaw sessions, your app's active Claude account pays for the check; your Codex or OpenAI account is never billed.

The card under the reply#

• Checking facts: the check is running.

• Checked: no unbacked claims found: nothing was flagged.

• Sent back N claims: the agent was asked to check them. Click the card to see each claim, why it was flagged and the quoted evidence.

• Watch only: N claims would have been sent back: the same list, with nothing sent.

• Fixed after check or Still unverified: N: what happened to the corrected reply.

• Not checked, with the reason: for example the daily limit is used up, plan usage is above your limit, or the check took too long. Your turn finished normally.

Each flag has a This flag was wrong button. Clicking it records that the flag was a false alarm; clicking again takes it back. It changes no session, sends nothing and costs nothing, and it is counted in Stats as Marked wrong.

See how it is doing#

Open Stats > Reliability and scroll to Fact check: checks run, how often claims were sent back (and how often Watch only would have), check time, flags marked wrong, flags by kind, checks that were skipped or failed, a row per engine, and a weekly correction rate chart with a line at the week the fact check was turned on. Corrections the check asked for are counted apart, so turning on Send back never makes agents look worse.