Observability sinks
The ObservabilityStrategy port — one required hot-path method that must be sync and must not throw. Compose several sinks into one, choose when delivery happens, and know exactly what a recording keeps and what it summarizes away.
Every run emits a typed AgentfootprintEvent stream. Where that stream goes
is a port with adapters, exactly like memory or hosting: the agent loop never
knows whether it is feeding CloudWatch, an OTel collector, a file, or nothing.
The port
interface ObservabilityStrategy extends BaseStrategy {
readonly capabilities: ObservabilityCapabilities;
exportEvent(event: AgentfootprintEvent): void;
}Two members of its own, six inherited — and only two of the eight are required:
| Member | Required | What it is |
|---|---|---|
name | ✅ | Registry key, lowercase-kebab ('datadog', 'agentcore'). Used to look it up and to de-dupe registrations |
capabilities | ✅ | { events?, logs?, traces?, metrics? } — declared, so a consumer can branch instead of guess |
exportEvent(event) | ✅ | The hot path. Translate to the vendor's wire format and ship |
flush?() | — | Drain whatever the hot path buffered. void for sync sinks, Promise<void> for async ones |
stop?() | — | Teardown: close clients, clear timers. Idempotent, and terminal — after it, events are dropped and there is no restart |
relevantEventTypes? | — | An event-type filter. Set it and the dispatcher only forwards those types |
validate?() | — | Config validator, called once at registration. Throw here rather than produce an empty dashboard |
_onError? | — | Error sink for this strategy's own failures. A property you assign after construction, not a constructor option |
Two laws govern exportEvent, and they are what make a bad sink survivable:
exportEventMUST be syncvoid. Buffer internally; drain inflush().
MUST NOT throw. Errors caught and routed to
_onErrorat the dispatch layer; one bad strategy never breaks the agent loop.
Who calls flush() and stop()
Three doors, since 8.12.0: the handle enable.* returns (handle.flush() /
handle.stop()), agent.shutdown(), and a standingAgent closing with its
default shutdown: 'flush'. agent.run() never flushes — that part is
unchanged, and it is why a batching exporter still needs one of the three at
process end.
Mounting one
import { agentcoreObservability } from 'agentfootprint/observe';
const handle = agent.enable.observability({
strategy: agentcoreObservability({ region: 'us-west-2', logGroupName: '/agentfootprint/assistant' }),
});ObservabilityEnableOptions is { strategy?, tier?, sampleRate?, detach?, flushOn? }.
No strategy means no telemetry — not a default one
enable.observability() called without a strategy returns a no-op handle and
attaches nothing. It does not silently fall back to
consoleObservability(): choosing a destination is an opinion the library does
not impose. Pass a strategy, or you get silence.
Axis 1 — which sink
All on agentfootprint/observe:
| Adapter | Ships to | Peer dep | Choose it when |
|---|---|---|---|
consoleObservability() | stdout | none | Local development; you want to see the stream |
cloudwatchObservability(opts) | CloudWatch Logs | @aws-sdk/client-cloudwatch-logs | You already read CloudWatch |
agentcoreObservability(opts) | CloudWatch Logs, in AgentCore's GenAI-Observability shape | @aws-sdk/client-cloudwatch-logs | A hosted AgentCore runtime, so agent events land beside the runtime's own telemetry |
xrayObservability(opts) | AWS X-Ray segment trees | @aws-sdk/client-xray | You trace with X-Ray. serviceName is required — a missing one is a TypeError |
otelObservability(opts) | Your OTel tracer, as spans + span events under gen_ai.* | @opentelemetry/api | Portability, or a collector you already run |
fileObservability(opts) | NDJSON on a local disk — one JSON.stringify(event) per line | none (node:fs) | There is no collector to ship to. A log shipper, jq or a person with grep reads the file already |
auditExport(opts) | A hash-chained bundle | none (node:crypto) | You need an evidence record, not a dashboard |
agentcoreObservability is a thin wrapper over the CloudWatch base — the only
difference is the strategy name. That is stated here rather than implied,
because it tells you the two rows have the same reliability, not merely similar
shapes.
otelObservability takes your tracer rather than bundling an SDK: OTel's SDK
is heavyweight and exporter-specific, and forcing one would defeat the
portability the standard exists for. When an injected tracer's spans do not
implement addEvent, the adapter falls back to flattened
${eventName}.${key} attributes — degraded (last-write-wins) but never silently
dropped.
Google Cloud — the same adapter, pointed at telemetry.googleapis.com
Google's Telemetry API accepts standard OTLP, and Google's own migration guide
recommends moving off the Cloud Trace exporter because its proprietary transform
can lose data. So there is no googleCloudTracer() factory and there is not going
to be one: it would construct a TracerProvider from packages you install either
way and attach a GoogleAuth header — about twenty lines of wiring behind four new
optional peer dependencies. otelObservability() already takes the tracer.
const auth = new GoogleAuth({ scopes: 'https://www.googleapis.com/auth/cloud-platform' });
const client = await auth.getClient();
const exporter = new OTLPTraceExporter({
url: 'https://telemetry.googleapis.com', // regional: telemetry.<REGION>.rep.googleapis.com
// TLS *and* renewable Google call credentials — not TLS plus copied headers.
credentials: credentials.combineChannelCredentials(
credentials.createSsl(),
credentials.createFromGoogleCredential(client),
),
});plus OTEL_RESOURCE_ATTRIBUTES=gcp.project_id=<PROJECT_ID> and
GOOGLE_CLOUD_QUOTA_PROJECT=<PROJECT_ID>. Google's console
renders the GenAI views from the OpenTelemetry GenAI semantic conventions —
the same gen_ai.* attributes this adapter already emits — once you opt in with
OTEL_SEMCONV_STABILITY_OPT_IN=gen_ai_latest_experimental. There is no
auto-instrumentation for @google/genai on npm, so agentfootprint's own provider
hooks are the instrumentation.
Take the pinned versions, and do not pass `tracer` on this one
The Google path has three sharp edges that a 2026-08 field trial found the hard
way — OTel 2.x moved span processors into the NodeTracerProvider constructor,
google-auth-library@11 hands @grpc/grpc-js a Headers object it cannot read
(so the exporter ships unauthenticated), and passing an explicit tracer to
otelObservability produced six disconnected traces with zero child spans. The
working shape registers the provider globally and omits tracer. Full
copy-paste recipe, with the exact versions:
Google Cloud & Gemini.
fileObservability rotates ONE generation, and nothing is bounded on the way out
maxBytes renames the file to <path>.1 — replacing any previous .1 — and
starts fresh. No .2, no compression, no schedule, no cross-process
coordination: it is a disk-safety ceiling, and retention belongs to the
log-management daemon you already run (omit maxBytes and the adapter never
renames anything, which is the right choice when logrotate owns the file).
Payloads are written verbatim, so narrow what reaches the strategy with
eventTypes / tier / sampleRate or a footprintjs RedactionPolicy — a
bounded record by construction is auditExport({ payloadMode: 'bounded' }).
An unwritable path is refused at construction, naming the path. Full detail
on On-premises & self-hosted.
auditExport is tamper-EVIDENT, not tamper-PROOF
Every record carries the SHA-256 of its own canonical serialization plus the hash
of the previous record, and verifyAuditBundle names the exact record that
broke. It does not prove provenance: an adversary holding the only copy can
recompute every hash from the mutation onward and present a self-consistent
forgery. Ship the bundle somewhere they do not control, and that gap closes.
Independently reproduced against a local harness — the cross-segment chain, 2026-08-13
auditExport's hash chain is contract-shaped and tested — independently
reproduced against a local harness, 2026-08-13. Two runs were captured as
separately drained segments under
payloadMode: 'bounded': 31 records each, 62 of 62 verified across the segment
boundary, segment 2's chain head equal to segment 1's final hash, an exact
prompt / tool-argument secret marker absent from all serialized output, and a
single edited event type caught by verifyAuditBundle at sequence 1 with a
hash mismatch.
The rung is stated rather than rounded up. That run was deterministic and local — two mock-provider runs in one process, verified in the same process (its own note: "local and deterministic, so they consumed no GCP credit"). No bundle has been written to, re-read from, or verified out of a live durable store from this repository, so this is not field-validated: the hashing law is proven, the shipping-it-somewhere-they-do-not-control half is still yours to prove. What the run does buy is that the chain held across a segment boundary under somebody else's hands, not the author's.
Two sentences from that run stay exactly where they were. The chain is
tamper-evident, not tamper-proof (above). And the strategy observes Agent
events, so ingress refusals — a 401 from identity.verify, a 429 from
admission — are not in the bundle, because they happen before a run exists.
standingAgent({ onIngressDecision }) is the separate
stream for those, and it deliberately does not join this chain.
Axis 2 — composition
composeObservability(children) returns one ObservabilityStrategy named
'compose', whose capabilities are the OR of its children's:
import { composeObservability, agentcoreObservability, otelObservability } from 'agentfootprint/observe';
agent.enable.observability({
strategy: composeObservability([
agentcoreObservability({ region: 'us-west-2', logGroupName: '/agentfootprint/assistant' }),
otelObservability({ tracer }),
]),
});exportEvent fans out to every child; flush() and stop() reach all of them.
One child failing does not stop the others — the passive-recorder rule. Siblings
exist for the other strategy kinds: composeCost, composeLiveStatus,
composeLens.
Axis 3 — delivery timing
detach decides when the sink's work happens relative to the agent loop:
detach | Behaviour | Choose it when |
|---|---|---|
| omitted | The strategy's hot path runs inline, in the loop | Cheap sinks; you want back-pressure to be visible |
{ driver, mode: 'forget' } | Scheduled off the loop; the handle is discarded | Pure telemetry — the common case |
{ driver, mode: 'join-later', onHandle } | Scheduled off the loop; every handle is delivered to onHandle so you can await it | You must know delivery finished (a test, a shutdown barrier) |
driver is required — there is no library default, because the right one is
an environment fact. Drivers come from footprintjs/detach:
microtaskBatchDriver (cross-runtime, the usual in-process pick),
setImmediateDriver (Node), setTimeoutDriver (configurable delay),
immediateDriver, plus factories createSendBeaconDriver (browser, survives
page-unload) and createWorkerThreadDriver.
import { microtaskBatchDriver } from 'footprintjs/detach';
agent.enable.observability({
strategy: agentcoreObservability({ region: 'us-west-2', logGroupName: '/agentfootprint/assistant' }),
detach: { driver: microtaskBatchDriver, mode: 'forget' }, // keep the loop unblocked
});Axis 4 — volume
tier is 'minimal' | 'standard' | 'firehose', default 'standard', and
filters which event types reach the strategy. sampleRate (0–1) thins what is
left.
tier is not a privacy control. It reduces volume, not sensitivity —
redactContent does not apply to it. If a payload must not leave the process,
filter or redact it, do not lower the tier.
relevantEventTypes on the strategy itself is the finer instrument: the
dispatcher will not even forward types outside that set.
Which conversation an event belongs to
EventMeta.sessionId (9.4.0) rides beside runId — one session produces many
runs, and an event needs to say which of each it is. The CloudWatch and AgentCore
adapters serialize the whole envelope, so it arrives without their knowing about
it.
await agent.run(message, { sessionId }); // or let standingAgent do it for youstandingAgent sets it on both run() and resume() from the request's own
session id. It is never derived, guessed, or defaulted to the runId: an
anonymous request has no session, and an absent key and an invented one are
different facts. Worked example: Exporters.
Who the run was for — the audit wire (9.11.0)
The stream has always said what happened and when. EventMeta.principal
and EventMeta.tenant are the who, and together the three are an audit
record rather than a debug log.
await agent.run(message, { identity: { tenant: 'acme', principal: 'alice@acme.test', conversationId } });Every event of that run carries meta.principal and meta.tenant.
They are stamped only from an identity a caller NAMED — run({ identity })
or run(input, { identity }), the same tuple memory and the permission gate
scope on. Three things deliberately do not produce them:
| Situation | principal / tenant |
|---|---|
run(msg, { identity: { tenant, principal, conversationId } }) | Stamped |
run(msg, { sessionId }) — the session-derived default | Absent |
run(msg) — anonymous | Absent |
A conversation id is not an actor
The run's internal identity is always populated: it defaults to
{ conversationId: '<runId>' }, or since 9.10.0 to { conversationId: sessionId }
on a session-bound run. Neither is a person. A sessionId in particular is
caller-supplied — anyone who can reach the host can send any string, including
somebody else's — so promoting it to "who did this" would produce an audit trail
that looks complete and names the wrong party. Absent is the honest answer, and
absent is what the meta says.
Which sinks carry it
Two kinds of adapter, and the difference decides the answer:
| Sink | Carries the actor | Why |
|---|---|---|
fileObservability | ✅ automatically | One JSON.stringify(event) per line — the whole envelope |
cloudwatchObservability | ✅ automatically | The log message is the serialized envelope |
agentcoreObservability | ✅ automatically | The same builder as CloudWatch |
auditExport | ✅ automatically | The meta rides the record, so the actor is inside the hash chain — editing who breaks the same verification as editing what |
otelObservability | ✅ placed deliberately | two agentfootprint.* span attributes — principal.id and tenant.id — on the invoke_agent run span |
xrayObservability | ✖ not mapped | Segment-mapping like OTel, but no attribute is placed. Said plainly rather than implied |
An envelope-serializing sink inherits any new meta field for free; a
span-mapping one does not, and a doc that assumed otherwise would have been
wrong about two of the six. OTel uses agentfootprint.* names rather than
enduser.id: the semantic convention for that attribute has moved once already,
and quietly claiming a convention we do not track is worse than a name that says
whose field it is.
What a recording keeps
recordRun(runner, options?) produces the canonical Recording —
{ snapshot, events, structure } — which is what an offline viewer replays.
| Option | Default | What it controls |
|---|---|---|
maxEvents | 10000 | Oldest events are dropped past this; the count is reported as droppedEvents |
boundaryDetail | 'full' | 'full' = every field, content included. 'lean' = boundary structure only, no captured payloads |
recordEmbeddings | false | Whether raw vectors survive the freeze |
Embeddings are summarized, not kept. By default every embedding /
embeddings field — in boundary payloads, in the snapshot's subflow results, in
tool results riding the event stream — is replaced with { dims, norm } when the
recording is frozen. dims is the dimensionality; norm is the L2 norm rounded
to four decimals — a checksum, not an operand.
The measurement behind the default: one retrieval turn's recording weighed
2.76 MB, about 1.1 MB of it embedding floats. Nothing that renders a recording
reads those floats, and the retrieval evidence a debugger actually needs — score,
passage, document, rejected candidates — carries no vectors and is untouched. Set
recordEmbeddings: true only when a consumer genuinely replays raw vectors
offline.
The same option exists on BoundaryRecorder, where it applies at capture
time rather than at freeze time.
Status
| Piece | Door | Status |
|---|---|---|
ObservabilityStrategy port, composeObservability | agentfootprint/observe | Shipped |
consoleObservability, auditExport / verifyAuditBundle | agentfootprint/observe | Shipped — no vendor SDK |
fileObservability | agentfootprint/observe | Field-validated — validated in an independent field trial, 2026-08 (Node 22: 30 asserted events written as NDJSON and parsed back, on a live agent run) |
otelObservability | agentfootprint/observe | Shipped — BYO tracer |
Cloud Trace / Cloud Logging via otelObservability + OTLP | agentfootprint/observe | A documented recipe, not a new adapter — field-validated in its corrected form, an independent field trial, 2026-08 |
cloudwatchObservability, agentcoreObservability, xrayObservability | agentfootprint/observe | Shipped; contract-mapped and injection-tested, command names pinned |
detach delivery, tier, sampleRate | — | Shipped |
recordRun + { dims, norm } embedding summary | agentfootprint/observe | Shipped |
EventMeta.sessionId | agentfootprint/events | Shipped (9.4.0) |
EventMeta.principal / EventMeta.tenant | agentfootprint/events | Shipped (9.11.0) — explicit identity only |
Actor on otelObservability run spans | agentfootprint/observe | Shipped (9.11.0) |
Actor on xrayObservability segments | — | Not mapped |
Next
- Identity & credentials — where the actor on the wire comes from
- Observability — the event taxonomy and the three workflows
- Exporters: AgentCore & OTEL — step-by-step wiring for both
- Offline replay — what a recording is for
- AWS & Bedrock AgentCore — the worked provider
- Google Cloud & Gemini — the OTLP recipe in full, and the rest of that column
Tools & gateways
The McpClient port — four members, three transports. Where a tool comes from is a transport decision (stdio, fixed HTTP headers, per-request vending, or your own signing fetch); what a tool is allowed to do is a governance decision, and they compose without knowing about each other.
AWS & Bedrock AgentCore
The first worked provider. Every AWS service agentfootprint adapts — Memory, Identity, Runtime hosting, Gateway MCP, Observability, Policy, S3 Vectors, Bedrock models and embedders — with its adapter, its door, its peer dependency, the SDK commands it dispatches, and its honest status.
