Recorder storage primitives

A recorder has two halves:

• Observer half — how it hears events. Implements one of ScopeRecorder / FlowRecorder / EmitRecorder / CombinedRecorder.
• Storage half — where it keeps data. A typed bookkeeping structure on the storage shelf.

This page is about the storage shelf. Three primitives, three different durability and indexing properties:

Primitive	Stores	Time scope	Memory
SequenceStore<T>	append-only ordered + keyed entries	durable across run	O(N events)
KeyedStore<T>	1:1 entry per runtimeStageId	durable across run	O(N steps)
BoundaryStateStore<TState>	transient bracket-scoped state	live; clears on stop	O(K active)

All three are imported from 'footprintjs/trace'.

Caution

v5 composition model — stores, not base classes. Under the “one purpose per recorder” rule, the storage shelf ships as concrete stores you own as a field (new SequenceStore<T>(), new KeyedStore<T>(), new BoundaryStateStore<TState>()) and compose with the observer interface you implement. The abstract base classes SequenceRecorder<T> / KeyedRecorder<T> / BoundaryStateTracker<TState> were removed in 7.0.0 — there is no inheritance path; composing the concrete stores is the only model. The examples below show that pattern.

When to pick which

"I need a permanent log of every event for time-travel"
                                 → SequenceStore<T>

"I want exactly one durable record per stage execution
 (e.g., metrics, costs, evals)"
                                 → KeyedStore<T>

"I need to know what's happening RIGHT NOW inside an
 in-flight matched-bracket interval (LLM stream partial,
 tool args streaming, agent turn state)"
                                 → BoundaryStateStore<TState>

Pick multiple when you need both durable AND live views — they compose freely.

Mental model

Recorder interfaces are observers. Storage primitives are bookkeeping shelves. A real recorder implements ONE observer interface AND owns ONE storage shelf, composing them in a single class.

This is exactly how the built-in recorders are built: CombinedNarrativeRecorder owns a sequence shelf AND implements CombinedRecorder. Same pattern; just pick the storage shelf that matches your data shape.

Primitive 1 — SequenceStore<T> (durable, ordered, keyed)

For event streams where order matters and you want per-step lookup AND ordered iteration.

import { SequenceStore } from 'footprintjs/trace';
import type { CombinedRecorder } from 'footprintjs';

interface AuditEntry {
  readonly runtimeStageId?: string;
  readonly type: 'read' | 'write' | 'decision';
  readonly detail: string;
}

class AuditRecorder implements CombinedRecorder {
  readonly id = 'audit';
  private readonly store = new SequenceStore<AuditEntry>();

  onWrite(e) { this.store.push({ runtimeStageId: e.runtimeStageId, type: 'write', detail: e.key }); }
  onDecision(e) {
    this.store.push({
      runtimeStageId: e.traversalContext?.runtimeStageId,
      type: 'decision',
      detail: `${e.decider} chose ${e.chosen}`,
    });
  }

  clear() { this.store.clear(); }
}

SequenceStore<T> methods: push, getAll / forEach / size, getByKey / keyCount, getEntryRanges() (O(1) range index for time-travel), aggregate, accumulate, getEntriesUpTo(visibleIds), clear.

Built-ins on this shelf (sequence-shaped): InOutRecorder, CombinedNarrativeRecorder.

Primitive 2 — KeyedStore<T> (durable, 1:1 by step)

For “exactly one record per stage execution” semantics — typical for metrics, costs, evaluation scores.

import { KeyedStore } from 'footprintjs/trace';
import type { ScopeRecorder } from 'footprintjs';

interface TokenEntry {
  readonly tokens: { readonly input: number; readonly output: number };
}

class TokenRecorder implements ScopeRecorder {
  readonly id = 'tokens';
  private readonly store = new KeyedStore<TokenEntry>();

  recordCall(stageId: string, usage: TokenEntry['tokens']) {
    this.store.set(stageId, { tokens: usage });
  }

  // Aggregate: grand total
  totalTokens() {
    return this.store.aggregate(
      (acc, e) => ({
        input: acc.input + e.tokens.input,
        output: acc.output + e.tokens.output,
      }),
      { input: 0, output: 0 },
    );
  }

  clear() { this.store.clear(); }
}

KeyedStore<T> methods: set / get / has / delete / getMap / values / size, aggregate, accumulate, filterByKeys, clear.

Built-ins (keyed-shaped): MetricRecorder, DebugRecorder, QualityRecorder.

Primitive 3 — BoundaryStateStore<TState> (transient, bracket-scoped)

For live transient state that exists ONLY during a matched [start, stop] event interval. Cleared on stop.

Use when: “Is something happening right now? What’s the partial value mid-stream?”

import { BoundaryStateStore } from 'footprintjs/trace';
import type { EmitEvent, EmitRecorder } from 'footprintjs';

interface LLMLiveState {
  readonly partial: string;
  readonly tokens: number;
}

class LiveLLMTracker implements EmitRecorder {
  readonly id = 'live-llm';
  private readonly store = new BoundaryStateStore<LLMLiveState>();

  onEmit(e: EmitEvent): void {
    if (e.name === 'demo.llm.start') {
      this.store.start(e.runtimeStageId, { partial: '', tokens: 0 });
    } else if (e.name === 'demo.llm.token') {
      const chunk = (e.payload as { content: string }).content;
      this.store.update(e.runtimeStageId, (s) => ({
        partial: s.partial + chunk,
        tokens: s.tokens + 1,
      }));
    } else if (e.name === 'demo.llm.end') {
      this.store.stop(e.runtimeStageId);
    }
  }

  // Public read API — O(1) at any moment
  isInFlight(): boolean { return this.store.hasActive; }
  getPartial(stageId: string): string {
    return this.store.get(stageId)?.partial ?? '';
  }

  clear() { this.store.clear(); }
}

const tracker = new LiveLLMTracker();
executor.attachEmitRecorder(tracker);
await executor.run();

// Read live state DURING the run from another async context:
tracker.isInFlight();           // true while llm.start↔llm.end is open
tracker.getPartial(rid);        // 'I will help you with that'

Algorithmic framing

This is the DFS bracket-sequence pattern — stack-frame state during a graph-traversal interval. The internal active Map is the open-brackets stack at any moment. Same shape used by Tarjan’s SCC algorithm, tree decomposition, and push-down automata.

Lifecycle contract — strict

Every store.start(key, ...) call MUST be paired with a store.stop(key) call. Failure to wire stop is a memory leak — the active map grows unboundedly and getAll() returns stale entries that look in-flight but aren’t. Common cause: your recorder wires start but forgets stop.

Tip

Dev-mode safety: call enableDevMode() at app startup. The store then warns at clear() time when residual active boundaries are detected, naming the leaked keys so you can find the missing wiring. Zero overhead in production.

What it does NOT do

• Time-travel queries. Transient state clears on stop. To answer “what was the state at past slider step N?”, snapshot the state at each emit into a separate SequenceStore<TState>.
• Run-wide aggregates. For totals/counts, use SequenceStore.aggregate() or KeyedStore.aggregate().
• Stage-level concerns. Use ScopeRecorder.onStageStart / ScopeRecorder.onStageEnd. This primitive operates at finer granularity (events emitted DURING a stage execution).

Concurrency / nesting

• Concurrent boundaries of the same kind (parallel branches with two LLM calls active at once) work correctly — keyed independently in the active map.
• Nested boundaries of different kinds (Agent boundary contains LLM boundary contains streaming) require separate store instances — one per kind. Compose multiple stores; each one tracks its own bracket pair.

Key convention

The key: string is whatever your recorder picks. Convention: use runtimeStageId when the boundary maps 1:1 to a stage execution — gives free interop with SequenceStore.getByKey, KeyedStore.get, findCommit / findLastWriter, and the rest of the trace ecosystem. Use a more granular key (e.g., toolCallId) only when there are multiple concurrent boundaries WITHIN one stage execution.

Choosing the right shelf — decision tree

1. Is the data DURABLE (kept after the run completes)?
     yes → step 2
     no  → BoundaryStateStore<TState>

2. Are there MULTIPLE entries per stage, and does ORDER matter?
     yes → SequenceStore<T>
     no  → KeyedStore<T>

Ninety-five percent of recorders fit one of the three. If yours doesn’t, file an issue — the storage shelf is intentionally small and we want to keep it that way.

Composing storage with observer interfaces

Same pattern for all three shelves — own the store as a field, implement the observer interface:

// SequenceStore + CombinedRecorder
class A implements CombinedRecorder { private store = new SequenceStore<MyEntry>(); ... }

// KeyedStore + ScopeRecorder
class B implements ScopeRecorder { private store = new KeyedStore<MyEntry>(); ... }

// BoundaryStateStore + EmitRecorder
class C implements EmitRecorder { private store = new BoundaryStateStore<MyState>(); ... }

Attach the same way:

executor.attachCombinedRecorder(new A());
executor.attachScopeRecorder(new B());
executor.attachEmitRecorder(new C());

Storage shelves don’t change the attach API. They just give your recorder a typed bookkeeping data structure with an idiomatic public read API and a clear clear() lifecycle.

Runnable example

A complete LiveLLMTracker example demonstrating bracket-scoped transient state:

npx tsx examples/runtime-features/data-recorder/06-boundary-state-tracker.ts

The example also doubles as an integration test in the suite — examples in examples/ run end-to-end on every release. It uses the store-based composition shown above (implements EmitRecorder + an owned BoundaryStateStore field) — the only model since 7.0.0.