RAG

Your support agent confidently tells a customer the refund window is 30 days. The actual policy says 14. The LLM is hallucinating from training data because you never gave it the source. RAG is how you stop the hallucination at its root — embed the user's question, retrieve the actual policy chunks, inject them into the system-prompt slot, and the LLM answers from the SOURCE instead of memory.

What RAG is

RAG = retrieval-augmented generation. Conceptually:

1. Embed the user's query into a vector
2. Search a vector store for top-K most-similar document chunks
3. Inject those chunks into the next LLM call's system-prompt slot, as one system message
4. The LLM answers using the retrieved chunks as context

In agentfootprint, RAG runs on the same machinery as defineMemory({ type: SEMANTIC, strategy: TOP_K }) — the same engine, the same three slots, the same agentfootprint.context.injected event. defineRAG differs from it in three ways that are not cosmetic, and each one is there because a corpus is not a conversation:

Define a retriever, attach to an agent

defineRAG({ id, store, embedder, description?, embedderId?, corpus?, topK?, threshold?, maxChars?, retrieval? }) returns a MemoryDefinition. agent.rag(definition) attaches it (alias for .memory(definition) — same plumbing, clearer intent). It throws at construction time if store lacks search(). Pair embedderId here with the same value passed to indexDocuments so a later embedder swap is filtered out of results:

const docs = defineRAG({  id: 'product-docs',  description: 'Product documentation chunks',  store,  embedder,  topK: 2,           // up to 2 most-relevant docs per query  threshold: 0.5,    // strict — drop weak matches});// Matches land in the SYSTEM-PROMPT slot as one system message carrying// every chunk as a `<source id=… doc=… score=…>` block the model can// cite. (`asRole` was removed in 7.20.0 — it was never read, so it// described a placement that never happened.)const agent = Agent.create({  provider: provider ?? mock({ reply: 'Refunds are processed within 3 business days.' }),  model: 'mock',  maxIterations: 1,})  .system('You answer support questions using the retrieved docs.')  .rag(docs)  .build();

The store is a MemoryStore with vector-search support (it must implement search() — defineRAG throws at construction time if it doesn't). InMemoryStore works for dev. Durable vector backends are not yet shipped — wire your own MemoryStore adapter against src/memory/store/types.ts until then.

The corpus namespace

A corpus does not belong to a conversation, so it does not read under the run's identity. corpus names the namespace it lives in, and it defaults to { conversationId: '_global' } — the same default indexDocuments writes to, so index with no options and retrieve with no options and the two meet.

For a per-tenant corpus, pass the same identity to both sides:

await indexDocuments(store, embedder, docs, { identity: { tenant: 'acme' } });
const acmeDocs = defineRAG({ id: 'docs', store, embedder, corpus: { tenant: 'acme' } });

If they ever disagree, you no longer find out by getting a vague answer: the retrieval reports corpusEmpty: true on agentfootprint.memory.retrieved, and the first time it happens in a process a warning names the namespace it searched.

Strict threshold semantics

threshold is strict. When no chunk meets the threshold, NO injection happens — the LLM gets no context and answers from its own weights. This is intentional: weak low-confidence chunks pollute the prompt and make a confident wrong answer MORE likely, not less. Nothing throws; the turn simply proceeds without context.

What changed in 8.8.0 is that "no context" is now a readable outcome instead of an absence. The near-misses and their scores are on the agentfootprint.memory.retrieved event, so the right threshold is a number you read off a run rather than one you guess:

agent.on('agentfootprint.memory.retrieved', (e) => {
  if (e.payload.admittedCount === 0) {
    console.log('nothing cleared', e.payload.threshold, '— best was', e.payload.candidates?.[0]);
  }
});

The right threshold is a property of the embedder, not of this library. The default 0.7 suits some score distributions and silently starves others:

The Titan numbers are one vendor's measured example of the general rule: read your embedder's actual score bands off a run before trusting any default. The rejected candidates and their scores are on every agentfootprint.memory.retrieved event, so the right threshold is a number you read, not one you guess — for Titan V2, ~0.5 separates its signal from its noise.

How much text, not just how many chunks — maxChars

topK bounds how MANY passages reach the prompt and says nothing about how long they are. A count bound is not a size bound, and the gap between them is where a corpus over-runs its slot with nothing but defaults on either side. Ten chunks cut by byHeading() off ordinary documentation measured 11,153 characters in a production deployment, against a systemPrompt slot whose default budget is 4,000.

Nothing truncates when that happens — the slot warns once and emits agentfootprint.context.budget_pressure, so the run is honest about the over-run. What it had no way to do was bound it. maxChars is that bound:

defineRAG({ id: 'docs', store, embedder, topK: 5, maxChars: 2000 });

The two numbers that meet, side by side — they live on different objects, which is exactly why the arithmetic between them is easy to miss:

Retrieved passages share that slot with the system prompt, steering, facts and skill bodies, so roughly half the slot is a sane starting budget: maxChars: 2000 against the 4,000-char default.

maxChars has no default, deliberately. Defaulting it would mean this release silently stops injecting passages that the last one injected — a retrieval regression that reads to a user as "the model doesn't know that", which is the failure class this library exists to make loud. And nothing here can know your slot budget: you may have raised contextBudget.systemPrompt. So the bound is yours to state, and the table above is the arithmetic.

The spend is recorded, never silent. The budget is spent in rank order and the tail is dropped — passages past it are refused with reason: 'over-char-budget', and the record carries maxChars and charsUsed:

agent.on('agentfootprint.memory.retrieved', (e) => {
  console.log(`${e.payload.charsUsed}/${e.payload.maxChars} chars`);
  for (const c of e.payload.candidates ?? []) {
    if (c.reason === 'over-char-budget') console.log('dropped for size:', c.id);
  }
});

Rank order and drop, never "skip the big one and take the next": skipping would silently reorder relevance by length, and a reader of the record could not tell a budget drop from a bad score. A budget smaller than the best-scoring passage therefore admits nothing — and says so, per candidate, instead of injecting half a passage.

maxChars composes with retrieval (unlike topK/threshold, which exclude it): the strategy picks the candidates, this bounds their size. It counts passage characters, not rendered prompt bytes — the <source …> wrapper and the block header are added afterwards by the formatter.

The three numbers, and the one place they collide

There is a third, at index time, and it is the one that fails silently. Read them together:

At the defaults the first two compose safely — 1,000 cut, at least 2,000 read. The trap opens when you raise the splitter's ceiling. byHeading({ maxChars: 2500 }) against a 2,000-character reading ceiling stores each chunk whole as the passage and indexes it by its opening: retrieval cannot find wording that is plainly visible in the <source> block the model is shown, and nothing throws. Measured in a production corpus: 6 of 26 chunks.

Since 9.1.0 the reading ceiling comes from the embedder itself (Embedder.maxInputChars — 2,000 for localEmbedder, 32,000 for the hosted ones), an explicit maxChunkChars still wins over it, and a run that clipped anything says so once on console.warn with the count and the fix. Full table on the indexing page.

Indexing — indexDocuments

Before queries can return chunks, the store needs documents. indexDocuments(store, embedder, docs, options?) is the seeding helper — embeds each doc (via embedder.embedBatch when available, else capped-concurrency single calls), then batches into store.putMany(). Returns the number of docs indexed. Each RagDocument is { id, content, metadata? } — or { id, text, metadata? }, since 8.19.0, because that is how a chunk from the rag door spells its passage and both are now read. A document carrying neither is refused before anything is embedded: an unrenderable passage and an absent one are different facts, and a citation wrapped around an empty body is the one that costs you a customer.

import { indexDocuments } from 'agentfootprint';

const count = await indexDocuments(store, embedder, [
  { id: 'doc1', content: 'Refund policy: 14 days from delivery for full refund.' },
  { id: 'doc2', content: 'Pro plan costs $20/month and includes priority support.', metadata: { topic: 'plans' } },
]);

The optional fourth argument is an IndexDocumentsOptions bag: identity, embedderId, tier, ttlMs, signal, and maxConcurrency (caps concurrent embed calls at 8 by default to avoid embedder rate limits; ignored when the embedder implements embedBatch):

await indexDocuments(store, embedder, docs, {
  identity: { tenant: 'acme' },   // scope the corpus to one tenant
  embedderId: 'openai-3-small',   // tag entries so a later embedder swap is filtered out
  maxConcurrency: 4,
});

Identity defaults to { conversationId: '_global' } — matching defineRAG's corpus default, so the plain path needs no argument on either side. Index per tenant and you must name the same tenant on the retriever; see The corpus namespace.

The corpus as a build artifact — exportCorpus / staticVectorStore

An immutable or serverless runtime loses its disk between invocations and often holds no embedding-API credentials — while the build machine has both. So build the index where the credentials live, ship it with the deploy, and serve it read-only where the process runs (8.20.0):

// build step (CI, cron, deploy hook) — credentials and durable disk live here
import { indexFolder, exportCorpus } from 'agentfootprint/rag';

const store = new InMemoryStore();
await indexFolder('./docs', { to: store, embedder });
writeFileSync('corpus.json', JSON.stringify(await exportCorpus(store)));

// runtime — no disk, no writes, no drift
import { staticVectorStore } from 'agentfootprint/memory';

const corpus = staticVectorStore(JSON.parse(readFileSync('corpus.json', 'utf8')), embedder);
const docs = defineRAG({ id: 'docs', store: corpus, embedder });

The bundle is plain JSON — { entries: [{ id, text, vector, metadata }], embedder: { id, dimensions }, namespace } — so it survives any transport a deploy already has: a file next to the code, a bundler import, a KV fetch. Three properties are load-bearing:

• Read-only, loudly. Every write method refuses with the fix named (re-export at build time, or importCorpus(store, bundle) into a writable store). A static corpus that silently accepted writes would lose them with the process.
• The wrong embedder is refused at LOAD. The bundle records the embedder id and dimensions it was built with; pass your runtime embedder as the second argument and a mismatch throws at startup — the same fingerprint rule the durable store enforces (dimensions always decide; ids decide only when both sides named themselves) — instead of surfacing as an empty retrieval at the first question.
• Entries are served in the exact shape the formatter reads — passage on value.content, provenance under value.metadata — so a bundled corpus renders citations identically to a live-indexed one.

importCorpus(store, bundle) is the inverse: seed a writable store from a bundle at boot, or migrate a corpus between machines without re-embedding (and re-billing) anything. The CLI can produce a bundle directly: npx agentfootprint-index ./docs --to ./corpus.json --embedder local.

Mocks-first development

mockEmbedder() is deterministic and free, and it is plumbing-only: it is a 32-dimension character-frequency hash, so it will happily rank an off-topic sentence above the right one. Use it to prove the pipeline runs; never to judge whether retrieval is any good.

Four real embedders ship, all from agentfootprint/providers:

Each one declares the longest input it reads whole, and the indexing doors use that instead of guessing — which is why the same corpus can be split into much larger chunks for a hosted embedder than for the on-device one.

Never mix two in one store: dimensions differ, and cross-model similarity scores are not comparable. Tag both sides with the same embedderId and the mismatch is filtered instead of silently scored.

Pairing a corpus with conversation memory

They are two registrations over two stores, and that separation is the point — one remembers what the user said, the other retrieves what the corpus says:

import { Agent, defineRAG } from 'agentfootprint';
import { defineMemory, MEMORY_TYPES, MEMORY_STRATEGIES, InMemoryStore } from 'agentfootprint/memory';

const agent = Agent.create({ provider })
  .rag(defineRAG({ id: 'product-docs', store: corpusStore, embedder }))
  .memory(defineMemory({
    id: 'chat',
    type: MEMORY_TYPES.EPISODIC,
    strategy: { kind: MEMORY_STRATEGIES.WINDOW, size: 10 },
    store: conversationStore,
  }))
  .build();

Why did the agent read this passage?

Every retrieval leaves a record, and it names what it rejected as well as what it used.

On the event stream — agentfootprint.memory.retrieved fires once per retrieval with every candidate, and agentfootprint.memory.attached once per chunk that reached the prompt:

agent.on('agentfootprint.memory.retrieved', (e) => {
  for (const c of e.payload.candidates ?? []) {
    console.log(c.admitted ? '✓' : '✗', c.id, c.score.toFixed(2), c.reason ?? '');
  }
});
// ✓ refunds.md#3   0.81
// ✓ security.pdf#7 0.72
// ✗ pricing.md#0   0.68  below-threshold

On each injection — one InjectionRecord per chunk rather than one per retrieval, so agentfootprint.context.injected carries source: 'rag', sourceId (the chunk id), retrievalScore, rankPosition and threshold for that one passage.

In the trace — the record is lifted onto root state as retrievalEvidence_<id>, so a backward slice from the answer reaches the passage instead of stopping at the memory subflow boundary.

Two orderings are recorded and they are not the same thing: rank is how the chunk scored, promptPosition is where the budget picker put it in the message. Under the default recency ordering the best-scoring chunk can land last.

Replacing the retrieval rule

topK + threshold are shorthand for the rule; retrieval is that rule spelled out. Passing both is refused — they could disagree, and the recording would then name a k the run did not use.

import { topK } from 'agentfootprint';

defineRAG({ id: 'docs', store, embedder, retrieval: topK({ k: 5, threshold: 0.55 }) });

A cross-encoder re-ranker and a diversity (MMR) selector are the next two adapters behind this same interface. Neither ships yet; the seam exists so that when they do, nothing else moves.

Reference — the retrieval record

Types for reading a retrieval back. The record types are on the main barrel (a consumer handling agentfootprint.memory.retrieved needs them); the rest are on agentfootprint/memory.

Anti-patterns

• Don't fall back to top-K-anyway when the threshold returns nothing. Read the rejected candidates and set a threshold that fits your embedder instead.
• Don't change embedders between writes and reads — entries are tagged with the embedder used at index time. A silent swap corrupts retrieval.
• Don't judge retrieval quality with mockEmbedder. It measures letter frequency, not meaning.
• Don't pass huge documents (50KB) — chunk first. RAG quality is dominated by chunk size and chunk-boundary choice, and localEmbedder truncates at 512 tokens regardless.

Next steps

• Memory guide — defineMemory covers all 4 types × 7 strategies including RAG
• Memory store adapters — RedisStore · AgentCoreStore · planned vector-capable backends