|
| 1 | +--- |
| 2 | +title: OpenTelemetry |
| 3 | +id: otel |
| 4 | +order: 4 |
| 5 | +description: "Emit vendor-neutral OpenTelemetry traces and metrics from every TanStack AI chat() call, following the OTel GenAI semantic conventions." |
| 6 | +keywords: |
| 7 | + - tanstack ai |
| 8 | + - opentelemetry |
| 9 | + - otel |
| 10 | + - observability |
| 11 | + - tracing |
| 12 | + - metrics |
| 13 | + - gen_ai |
| 14 | + - semantic conventions |
| 15 | +--- |
| 16 | + |
| 17 | +The `otelMiddleware` factory wires TanStack AI into your existing OpenTelemetry setup. Every `chat()` call produces a root span, one child span per agent-loop iteration, and one grandchild span per tool call — all with [GenAI semantic-convention attributes](https://opentelemetry.io/docs/specs/semconv/gen-ai/). It also records GenAI token and duration histograms when a `Meter` is provided. |
| 18 | + |
| 19 | +## Setup |
| 20 | + |
| 21 | +Install `@opentelemetry/api` — it's an optional peer dependency of `@tanstack/ai`: |
| 22 | + |
| 23 | +```bash |
| 24 | +pnpm add @opentelemetry/api |
| 25 | +``` |
| 26 | + |
| 27 | +Wire up your OTel SDK however you already do (e.g. `@opentelemetry/sdk-node`). Then pass a `Tracer` (and optionally a `Meter`) into the middleware. The OTel middleware lives on its own subpath — importing it never affects users who don't need OTel: |
| 28 | + |
| 29 | +```ts |
| 30 | +import { chat } from '@tanstack/ai' |
| 31 | +import { otelMiddleware } from '@tanstack/ai/middlewares/otel' |
| 32 | +import { openaiText } from '@tanstack/ai-openai/adapters' |
| 33 | +import { trace, metrics } from '@opentelemetry/api' |
| 34 | + |
| 35 | +const otel = otelMiddleware({ |
| 36 | + tracer: trace.getTracer('my-app'), |
| 37 | + meter: metrics.getMeter('my-app'), |
| 38 | +}) |
| 39 | + |
| 40 | +const result = await chat({ |
| 41 | + adapter: openaiText('gpt-4o'), |
| 42 | + messages: [{ role: 'user', content: 'hi' }], |
| 43 | + middleware: [otel], |
| 44 | + stream: false, |
| 45 | +}) |
| 46 | +``` |
| 47 | + |
| 48 | +## What gets emitted |
| 49 | + |
| 50 | +### Spans |
| 51 | + |
| 52 | +```text |
| 53 | +chat gpt-4o (root, kind: INTERNAL) |
| 54 | +├── chat gpt-4o #0 (iteration, kind: CLIENT) |
| 55 | +│ ├── execute_tool get_weather |
| 56 | +│ └── execute_tool get_time |
| 57 | +└── chat gpt-4o #1 (iteration, kind: CLIENT) |
| 58 | +``` |
| 59 | + |
| 60 | +Iteration spans are numbered (`#0`, `#1`, ...) so distinct iterations of the same chat are easy to pick apart in trace viewers. |
| 61 | + |
| 62 | +### Attribute reference |
| 63 | + |
| 64 | +| Level | Attribute | Value | |
| 65 | +| --- | --- | --- | |
| 66 | +| root / iteration | `gen_ai.system` | `openai`, `anthropic`, ... | |
| 67 | +| iteration | `gen_ai.operation.name` | `chat` | |
| 68 | +| root / iteration | `gen_ai.request.model` | requested model | |
| 69 | +| iteration | `gen_ai.response.model` | actual model | |
| 70 | +| iteration | `gen_ai.request.temperature` | from config | |
| 71 | +| iteration | `gen_ai.request.top_p` | from config | |
| 72 | +| iteration | `gen_ai.request.max_tokens` | from config | |
| 73 | +| iteration | `gen_ai.usage.input_tokens` | per iteration | |
| 74 | +| iteration | `gen_ai.usage.output_tokens` | per iteration | |
| 75 | +| iteration | `gen_ai.response.finish_reasons` | `[stop]`, `[tool_calls]`, ... | |
| 76 | +| root | `gen_ai.usage.input_tokens` | rolled up | |
| 77 | +| root | `gen_ai.usage.output_tokens` | rolled up | |
| 78 | +| root | `tanstack.ai.iterations` | iteration count | |
| 79 | +| tool | `gen_ai.tool.name` | tool name | |
| 80 | +| tool | `gen_ai.tool.call.id` | tool call id | |
| 81 | +| tool | `gen_ai.tool.type` | `function` | |
| 82 | +| tool | `tanstack.ai.tool.outcome` | `success` / `error` | |
| 83 | + |
| 84 | +### Metrics |
| 85 | + |
| 86 | +Two GenAI-standard histograms: |
| 87 | + |
| 88 | +- `gen_ai.client.operation.duration` (seconds) — recorded **once per `chat()` call**, covering all agent-loop iterations and tool execution. On error or abort the record carries an `error.type` attribute (the thrown error's `name`, or `"cancelled"` for aborts). |
| 89 | +- `gen_ai.client.token.usage` (tokens) — recorded **once per iteration** (two records: input and output), tagged with `gen_ai.token.type`. |
| 90 | + |
| 91 | +Both `gen_ai.response.id` and `gen_ai.response.model` are deliberately excluded from metric attributes to keep cardinality low (per-request custom-model names and request IDs would blow up the series set). |
| 92 | + |
| 93 | +## Privacy: capturing prompts and completions |
| 94 | + |
| 95 | +By default, only metadata lands on spans. To record prompt and completion content, set `captureContent: true`. Content is captured as OTel span events following the GenAI convention: |
| 96 | + |
| 97 | +- `gen_ai.user.message`, `gen_ai.system.message`, `gen_ai.assistant.message`, `gen_ai.tool.message`, `gen_ai.choice` |
| 98 | + |
| 99 | +Pass a `redact` function to strip PII before anything is recorded: |
| 100 | + |
| 101 | +```ts |
| 102 | +otelMiddleware({ |
| 103 | + tracer, |
| 104 | + captureContent: true, |
| 105 | + redact: (text) => text.replace(/\b\d{3}-\d{2}-\d{4}\b/g, '[SSN]'), |
| 106 | +}) |
| 107 | +``` |
| 108 | + |
| 109 | +If `redact` throws, the middleware writes the literal sentinel `"[redaction_failed]"` into the span event and logs a warning — it never falls back to the raw content. This is the load-bearing invariant for users who ship traces to third-party backends: a broken redactor should shut off capture, not leak prompts. |
| 110 | + |
| 111 | +Accumulated assistant text (the `gen_ai.choice` event) is capped at `maxContentLength` characters (default `100 000`); longer completions are truncated with a trailing `"…"` marker. |
| 112 | + |
| 113 | +Multimodal content (images, audio, video, documents) is represented as placeholder strings (`[image]`, `[audio]`, ...) to preserve message order without dumping binary data onto spans. Use `onSpanEnd` if you need richer multimodal capture. |
| 114 | + |
| 115 | +Prompt/system/user message events fire from `onConfig` at the start of every iteration, which means the full conversation history (as the adapter will re-send it) is re-emitted on each iteration span. This mirrors what the provider actually sees on the wire. |
| 116 | + |
| 117 | +## Extension points |
| 118 | + |
| 119 | +All four extensions are optional. Each wraps user code in try/catch — a thrown callback becomes a log line, never a broken chat. |
| 120 | + |
| 121 | +### `spanNameFormatter(info)` |
| 122 | + |
| 123 | +Override default span names. `info.kind` is `'chat' | 'iteration' | 'tool'`. |
| 124 | + |
| 125 | +```ts |
| 126 | +otelMiddleware({ |
| 127 | + tracer, |
| 128 | + spanNameFormatter: (info) => |
| 129 | + info.kind === 'tool' ? `tool:${info.toolName}` : `chat:${info.ctx.model}`, |
| 130 | +}) |
| 131 | +``` |
| 132 | + |
| 133 | +### `attributeEnricher(info)` |
| 134 | + |
| 135 | +Add custom attributes to every span. Fires once per span. |
| 136 | + |
| 137 | +```ts |
| 138 | +otelMiddleware({ |
| 139 | + tracer, |
| 140 | + attributeEnricher: () => ({ |
| 141 | + 'tenant.id': getCurrentTenant(), |
| 142 | + }), |
| 143 | +}) |
| 144 | +``` |
| 145 | + |
| 146 | +### `onBeforeSpanStart(info, options)` |
| 147 | + |
| 148 | +Mutate `SpanOptions` immediately before `tracer.startSpan(...)`. Useful for adding links, custom start times, or extra default attributes. |
| 149 | + |
| 150 | +### `onSpanEnd(info, span)` |
| 151 | + |
| 152 | +Fires just before every `span.end()`. Common uses: record custom events, emit per-tool metrics via your own `Meter`. |
| 153 | + |
| 154 | +```ts |
| 155 | +const toolDuration = meter.createHistogram('tool.duration') |
| 156 | +otelMiddleware({ |
| 157 | + tracer, |
| 158 | + onSpanEnd: (info, span) => { |
| 159 | + if (info.kind === 'tool') { |
| 160 | + // span is still recording; read timestamps from your own store if needed |
| 161 | + toolDuration.record(1, { 'tool.name': info.toolName }) |
| 162 | + } |
| 163 | + }, |
| 164 | +}) |
| 165 | +``` |
| 166 | + |
| 167 | +## Related |
| 168 | + |
| 169 | +- [Middleware](./middleware) — the lifecycle this middleware hooks into |
| 170 | +- [Debug Logging](./debug-logging) — quick console-output diagnostics, complementary to OTel |
| 171 | +- [Observability](./observability) — TanStack AI's built-in event client |
0 commit comments