Build

Names and numbers from evidence

Every number, identifier and name in the final answer must appear in a tool result the run really read. If one does not, the model typed it rather than read it — a deterministic check, no second model.

A storage engineer asks which array port is affected. The agent answers with a port row: alias SHPMAXDLVAP001-FA0, FCID 0xef0101. Both look exactly like the real thing. Neither appears in any tool result from that turn — the model typed them. That is a real answer from a real production run, and nothing in the conversation could have caught it. The framework watched every tool result land, so it can.

.namesAndNumbersFromEvidence() requires every number, identifier and name in the final answer to appear in a tool result the run actually read.

const agent = Agent.create({ provider, model })
  .tool(showInterfaceStatus)
  .tool(showFlogi)
  .namesAndNumbersFromEvidence({ posture: 'guard' })
  .build();

const answer = await agent.run({ message: 'which array port is affected?' });

What it is — and what it provably is not

It is a fabrication detector, not a correctness judge.

It catches invented values. It cannot catch a false claim assembled from real values: "fc1/3 is healthy" when the data says the port is down uses entirely grounded tokens — fc1/3 is in the evidence, "healthy" is a word — and the check passes it without a murmur. So does "the outage started at 08:15" when 08:15 is a timestamp belonging to a different port. If you read this as a hallucination check you will trust it for the one thing it cannot do.

It is also conservative in the other direction. Small numbers, all-letters names and quantities with units (32G, 47 flaps, 892 CRC errors) are not examined, because a false accusation costs a real turn — and on a weak model a false accusation triggers exactly the retry loop this library exists to remove. A missed fabrication is a miss; a refused good answer is damage.

Why it is deterministic

No second model, no embedding, no LLM judge. The check is set membership over normalized tokens: it costs microseconds and answers identically on every run.

That constraint is the point. This library's thesis is that structure lets a smaller model perform like a bigger one — so a guard that needed a bigger model to police the small one would invert the whole value proposition, and would fail precisely where the small model is deployed: offline, cheap, fast.

The three postures

Same three words as .skillGraph({ strictness }) and a separate option, because routing authority and evidence discipline are different decisions.

postureWhat happens
'assist' (default)Record and flag. The answer goes out unchanged — you learn how often it happens before you act on it.
'guard'The unsupported values are named back to the model, which gets one more ordinary turn. Survivors ship flagged. Recommended for weaker models.
'rails'The same one revision, then run() raises UnsupportedValuesError rather than return an answer that still carries them.

guard is a branch of the ReAct loop, not a special mode: the correction is one more ordinary turn, with its own iteration_start / llm_start bracket and its own cost.tick — and the tools are still on the wire, so the model can go and fetch the value it guessed at. One revision per turn, latched: a model that cannot ground a value on its second try will not on its fifth.

// guard, in the run's own record:
// route_decided { chosen: 'evidence-recheck' }
// evidence_checked { action: 'revision-asked', unsupported: [ … ] }
// route_decided { chosen: 'final' }
// evidence_checked { action: 'grounded', afterRevision: true }   ← it worked

What counts as evidence

  • Tool results — the role: 'tool' turns of the conversation. That is the whole corpus. Results that are JSON are walked structurally (every key, every leaf) rather than searched as text, so a value that merely appears inside an unrelated field does not read as grounded.
  • Tool call arguments do not count. The model typed those; grounding a value because the model passed it to a tool would let any invention launder itself through one failed lookup.
  • The model's own earlier answers do not count, for the same reason.

Values the user supplied are exempt without being declared: this turn's message, the conversation's user and system turns, and the composed system prompt (base prompt, skill bodies, retrieved passages). The user gave them, so they were not invented.

Spelling differences are normalised on both sides, which is where a naive implementation produces its worst false positives: 41,200 in prose matches the JSON number 41200, 2048.0 matches 2048, and 0xef0101 matches ef0101.

Teaching it your identifiers

The default extractor guesses from digits and punctuation. It cannot know that ORD-4471 is an order number or that a WWN is a WWN. Declare the shapes and they are checked by name — shapes composes with the defaults rather than replacing them, and exempt removes what you know is safe.

.namesAndNumbersFromEvidence({
  posture: 'guard',
  shapes: [
    { name: 'wwn', match: /(?:[0-9a-f]{2}:){7}[0-9a-f]{2}/ },
    { name: 'order', match: /ord-\d{4,}/ },
  ],
  exempt: ['v9.35.0', /^build-\d+$/],
  minDigits: 4,   // when a BARE number stops being prose. Default 4.
})

A NamesAndNumbersOptions bag: posture (EvidencePosture), shapes (EvidenceShape[] — each a name plus a match pattern, anchored to a whole token), exempt, and minDigits.

Reading the record

Every judgement fires agentfootprint.agent.evidence_checked — whatever the posture and whatever the outcome, so a debugger can show the answer, the values and whether the revision fixed them.

agent.on('agentfootprint.agent.evidence_checked', (e) => {
  // action: 'grounded' | 'revision-asked' | 'flagged' | 'refused'
  log.info({ action: e.payload.action, values: e.payload.unsupported });
});

After the run, agent.unsupportedValues() returns the terminal verdict — the flagged UnsupportedValue[], whether a revision was spent (revised), and whether the answer was withheld (refused) — or undefined when every value was grounded.

Under 'rails' the run raises UnsupportedValuesError (an UnsupportedValuesContext: the values, the candidate count, revised). The error names the values and says what would satisfy the check; the refused answer is not carried on it and stays in the commit log under whatever redaction the run configured.

try {
  await agent.run({ message: 'which array port is affected?' });
} catch (e) {
  if (e instanceof UnsupportedValuesError) {
    log.warn({ invented: e.values.map((v) => v.value) });
  } else throw e;
}

The correction the loop sends is an authored frame followed by the quoted values; EVIDENCE_CHECK_FRAME_PREFIX is exported so a transcript reader (or a test) can recognise it without matching on prose.

Composition

  • With .outputSchema(): the schema is judged first — an answer with the wrong shape is about to be replaced wholesale, so grounding it would pay for the same turn twice. The evidence gate judges what the schema let stand, and an answer that exhausted its schema retries is not judged at all.
  • With .reliability(): no collision. Reliability governs what one call does before a response is committed; this governs an answer that was committed.
  • Unused, it costs nothing: an agent without the option mounts no branch, writes no state, and emits no event.

When to reach for it

Reach for 'assist' on any agent whose answers carry identifiers a human will act on — port names, order numbers, account ids, serials — and read the events for a week. Move to 'guard' when the numbers say you should, and especially when the model is a small one: naming the values back is the cheapest structural help a weak model gets. Reach for 'rails' only where an answer carrying an invented identifier is worse than no answer at all.

On this page