Skip to content

Commit c1fd96f

Browse files
feat(ai): debug logging system across all activities and adapters (#467)
* feat(ai): add Logger, DebugCategories, DebugConfig, DebugOption types * fix(ai): relocate logger type tests to tests/ for vitest discovery * docs(ai): add JSDoc to logger types and tighten type tests * feat(ai): add ConsoleLogger default logger implementation * feat(ai): add InternalLogger with per-category filtering and prefix * feat(ai): add resolveDebugOption normalizing DebugOption to InternalLogger * feat(ai): export Logger types publicly and InternalLogger via /adapter-internals subpath * feat(ai): thread InternalLogger through TextEngine, MiddlewareRunner, and TextOptions * feat(ai-openai): emit request/provider/errors logs via InternalLogger in text adapter * feat(ai-anthropic): emit request/provider/errors logs via InternalLogger in text adapter * feat(ai-gemini): emit request/provider/errors logs via InternalLogger in text adapter * feat(ai-grok): emit request/provider/errors logs via InternalLogger in text adapter * feat(ai-groq): emit request/provider/errors logs via InternalLogger in text adapter * feat(ai-ollama): emit request/provider/errors logs via InternalLogger in text adapter * feat(ai-openrouter): emit request/provider/errors logs via InternalLogger in text adapter * fix(ai): remove redundant top-level logger from StructuredOutputOptions and normalize provider log key * feat(ai-openai): emit logging events in summarize adapter * feat(ai-anthropic): emit logging events in summarize adapter * feat(ai-gemini): emit logging events in summarize adapter * feat(ai-grok): emit logging events in summarize adapter * feat(ai-ollama): emit logging events in summarize adapter * feat(ai-openrouter): emit logging events in summarize adapter * feat(ai): resolve debug option and thread InternalLogger through summarize() * feat(ai-openai): emit logging events in image adapter * feat(ai-gemini): emit logging events in image adapter * feat(ai-grok): emit logging events in image adapter * feat(ai-openrouter): emit logging events in image adapter * feat(ai-fal): emit logging events in image adapter * feat(ai): resolve debug option and thread InternalLogger through generateImage() * feat(ai-openai): emit logging events in video adapter * feat(ai-fal): emit logging events in video adapter * feat(ai): resolve debug option and thread InternalLogger through generateVideo() * feat(ai-openai): emit logging events in tts adapter * feat(ai-gemini): emit logging events in tts adapter * feat(ai): resolve debug option and thread InternalLogger through generateSpeech() * feat(ai-openai): emit logging events in transcription adapter * feat(ai): resolve debug option and thread InternalLogger through generateTranscription() * feat(ai-openai): emit logging events in realtime adapter * feat(ai-elevenlabs): emit logging events in realtime adapter * test(ai): add integration tests for debug logging across activities * fix(ai): remove redundant nullish coalescing on non-nullable result fields * test(e2e): debug logging emits expected prefixes for chat() * docs(advanced): add debug logging guide * docs(observability): cross-link to debug logging guide * docs(middleware): cross-link to debug logging guide * docs(nav): add Debug Logging entry to Advanced section * fix(ai): migrate remaining console.warn calls in realtime adapters to logger.errors * docs(advanced): fix stray comment syntax in debug-logging guide * ci: apply automated fixes * fix: resolve eslint and knip failures from debug logging PR - Remove unnecessary optional chains and nullish coalescing on required messages array across adapter log lines - Remove unnecessary fallback on non-nullable chunk/event types - Reorder imports in elevenlabs realtime types to satisfy import/first - Delete unused packages/typescript/ai/src/logger/index.ts barrel (public surface is re-exported from src/index.ts) * ci: apply automated fixes * fix(ai): suppress debug logs for internal devtools middleware The devtools middleware is injected automatically by chat() and is already excluded from aiEventClient instrumentation via shouldSkipInstrumentation. Its per-hook logger.middleware / logger.config calls were still firing though, flooding the [tanstack-ai:middleware] category with internal plumbing. Move those calls inside the same skip gate so debug output only reflects user-provided middleware. * feat(ai): pretty-print deeply nested meta in ConsoleLogger on Node Debug logs surface raw provider chunks whose nested structures (usage, output, reasoning, tools, response payloads) were being truncated to [Object] / [Array] because Node's default console formatting stops at depth 2. ConsoleLogger now lazily loads node:util and runs meta through inspect({ depth: null }) on Node so the entire structure renders. Browsers still get the raw object for interactive DevTools inspection. * ci: apply automated fixes * refactor(ai): use console.dir with depth:null instead of util.inspect console.dir is the purpose-built native API for depth-unlimited object inspection. It takes the same {depth, colors} options natively on Node and is a no-op/interactive-tree in browsers, so we get the expanded output in both environments without any dynamic import dance around node:util. * feat(ai): prefix each debug category with an emoji marker Makes it trivial to visually scan dense streaming logs — each category tag is now bracketed by its own emoji on both sides, e.g. '📨 [tanstack-ai:output] 📨 ...'. Mapping: request=📤, provider=📥, output=📨, middleware=🧩, tools=🔧, agentLoop=🔁, config=⚙️, errors=❌. Tests that asserted on the raw tag via startsWith were switched to includes so they remain robust to prefix changes. * ci: apply automated fixes * test: remove debug logging e2e spec Library-level unit tests in the @tanstack/ai test suite already cover the debug logging behaviour (logger wiring, category resolution, console.dir formatting, emoji prefixing). An e2e round-trip added no independent coverage, so drop the spec, its API route, its fixture, and the now-stale routeTree entry. * chore: add changesets for debug logging - @tanstack/ai: minor — new debug option on every activity, Logger / ConsoleLogger / DebugOption public surface, @tanstack/ai/adapter-internals subpath, emoji-prefixed category tags, console.dir-based meta formatting - All provider adapters: patch — wire adapters through the InternalLogger so request/provider/errors flow through the structured logger; drop leftover console.* calls in adapter catch blocks * ci: apply automated fixes * fix(ai): swallow user-logger exceptions and ship debug-logging skill Wrap the user-supplied Logger calls inside InternalLogger.emit in a try/catch so an exception from the injected logger never masks the real error that triggered the log call (e.g. a provider SDK failure inside the chat stream). Also ship a new ai-core/debug-logging skill under packages/typescript/ai/skills/ so agents can discover how to toggle debug logging on/off, narrow it per category, and pipe it into a custom logger. * docs(ai): document Logger try/catch guarantee in debug-logging guide * docs(ai): fix DebugOption type and add realtime note in debug-logging skill - Show actual `DebugOption = boolean | DebugConfig` type; describe the omitted-field behavior as a resolution rule rather than a type-arm. - Replace the misleading "when any flag is set" comment; flags default to true whenever a DebugConfig is passed. - Acknowledge that provider realtime session adapters (openaiRealtime, elevenlabsRealtime) also accept the same debug option. * test(ai-openrouter): add missing logger field to structuredOutput test PR #484 on main added a new structuredOutput test that didn't pass a logger in chatOptions. After merging main into this branch, the new required logger field on TextOptions (from the debug-logging PR) tripped the typecheck. Pass the silent testLogger already defined at the top of the file to match the pattern used by every other test in this file. --------- Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
1 parent 19d09f5 commit c1fd96f

70 files changed

Lines changed: 3315 additions & 583 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.
Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
---
2+
'@tanstack/ai-openai': patch
3+
'@tanstack/ai-anthropic': patch
4+
'@tanstack/ai-gemini': patch
5+
'@tanstack/ai-ollama': patch
6+
'@tanstack/ai-openrouter': patch
7+
'@tanstack/ai-grok': patch
8+
'@tanstack/ai-groq': patch
9+
'@tanstack/ai-elevenlabs': patch
10+
'@tanstack/ai-fal': patch
11+
---
12+
13+
Wire each adapter's text, summarize, image, speech, transcription, and video paths through the new `InternalLogger` from `@tanstack/ai/adapter-internals`: `logger.request(...)` before each SDK call, `logger.provider(...)` for every chunk received, and `logger.errors(...)` in catch blocks. Migrates all pre-existing ad-hoc `console.*` calls in adapter catch blocks (including the OpenAI and ElevenLabs realtime adapters) onto the structured logger. No adapter factory or config-shape changes.

.changeset/debug-logging-ai.md

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,22 @@
1+
---
2+
'@tanstack/ai': minor
3+
---
4+
5+
**Pluggable debug logging across every activity.** `chat`, `summarize`, `generateImage`, `generateVideo`, `generateSpeech`, and `generateTranscription` now accept a `debug?: DebugOption` that turns on structured per-category logs (`request`, `provider`, `output`, `middleware`, `tools`, `agentLoop`, `config`, `errors`).
6+
7+
```ts
8+
chat({ adapter, messages, debug: true }) // all categories on
9+
chat({ adapter, messages, debug: false }) // silent (incl. errors)
10+
chat({ adapter, messages, debug: { middleware: false } }) // all except middleware
11+
chat({ adapter, messages, debug: { logger: pino } }) // route to a custom logger
12+
```
13+
14+
Additions:
15+
16+
- New `Logger` interface (`debug` / `info` / `warn` / `error`) and default `ConsoleLogger` that routes to matching `console.*` methods and prints nested `meta` via `console.dir(meta, { depth: null, colors: true })` so streamed provider payloads render in full.
17+
- New `DebugCategories` / `DebugConfig` / `DebugOption` public types.
18+
- New internal `@tanstack/ai/adapter-internals` subpath export exposing `InternalLogger` + `resolveDebugOption` so provider adapters can thread logging without leaking internals on the public surface.
19+
- Each log line is prefixed with an emoji + `[tanstack-ai:<category>]` tag so categories are visually distinguishable in dense streams. Errors log unconditionally unless explicitly silenced.
20+
- `TextEngine`, `MiddlewareRunner`, and every activity entry point thread a resolved `InternalLogger` through the pipeline — no globals, concurrent calls stay independent.
21+
- Exceptions thrown by a user-supplied `Logger` implementation are swallowed so they never mask the real error that triggered the log call.
22+
- New `ai-core/debug-logging` skill shipped under `packages/typescript/ai/skills/` so agents can discover how to toggle, narrow, and redirect debug output.

docs/advanced/debug-logging.md

Lines changed: 163 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,163 @@
1+
---
2+
title: Debug Logging
3+
id: debug-logging
4+
order: 3
5+
description: "Turn on structured, category-toggleable debug logging to see every chunk, middleware transform, and tool call inside TanStack AI."
6+
keywords:
7+
- tanstack ai
8+
- debug
9+
- logging
10+
- logger
11+
- pino
12+
- troubleshooting
13+
- chunks
14+
- middleware debugging
15+
---
16+
17+
# Debug Logging
18+
19+
You have a `chat()` that isn't behaving as expected — a missing chunk, a middleware that doesn't seem to fire, a tool call with wrong args. By the end of this guide, you'll have turned on debug logging and will see every chunk, middleware transform, and tool call flowing through your call.
20+
21+
## Turn it on
22+
23+
Add `debug: true` to any activity call:
24+
25+
```typescript
26+
import { chat } from "@tanstack/ai";
27+
import { openaiText } from "@tanstack/ai-openai";
28+
29+
const stream = chat({
30+
adapter: openaiText("gpt-4o"),
31+
messages: [{ role: "user", content: "Hello" }],
32+
debug: true,
33+
});
34+
```
35+
36+
Every internal event now prints to the console with a `[tanstack-ai:<category>]` prefix:
37+
38+
```
39+
[tanstack-ai:request] activity=chat provider=openai model=gpt-4o messages=1 tools=0 stream=true
40+
[tanstack-ai:agentLoop] run started
41+
[tanstack-ai:provider] provider=openai type=response.output_text.delta
42+
[tanstack-ai:output] type=TEXT_MESSAGE_CONTENT
43+
...
44+
```
45+
46+
## Narrow what's printed
47+
48+
Pass a `DebugConfig` object instead of `true`. Every unspecified category defaults to `true`, so toggle by setting specific flags to `false`:
49+
50+
```typescript
51+
chat({
52+
adapter: openaiText("gpt-4o"),
53+
messages,
54+
debug: { middleware: false }, // everything except middleware
55+
});
56+
```
57+
58+
If you want to see ONLY a specific set of categories, set the rest to `false` explicitly. Errors default to `true` — keep them on unless you really want total silence:
59+
60+
```typescript
61+
chat({
62+
adapter: openaiText("gpt-4o"),
63+
messages,
64+
debug: {
65+
provider: true,
66+
output: true,
67+
middleware: false,
68+
tools: false,
69+
agentLoop: false,
70+
config: false,
71+
errors: true, // keep errors on — they're cheap and important
72+
request: false,
73+
},
74+
});
75+
```
76+
77+
## Pipe into your own logger
78+
79+
Pass a `Logger` implementation and all debug output flows through it instead of `console`:
80+
81+
```typescript
82+
import type { Logger } from "@tanstack/ai";
83+
import pino from "pino";
84+
85+
const pinoLogger = pino();
86+
const logger: Logger = {
87+
debug: (msg, meta) => pinoLogger.debug(meta, msg),
88+
info: (msg, meta) => pinoLogger.info(meta, msg),
89+
warn: (msg, meta) => pinoLogger.warn(meta, msg),
90+
error: (msg, meta) => pinoLogger.error(meta, msg),
91+
};
92+
93+
chat({
94+
adapter: openaiText("gpt-4o"),
95+
messages,
96+
debug: { logger }, // all categories on, piped to pino
97+
});
98+
```
99+
100+
The default logger is exported as `ConsoleLogger` if you want to wrap it:
101+
102+
```typescript
103+
import { ConsoleLogger } from "@tanstack/ai";
104+
```
105+
106+
### Your `Logger` is wrapped in a try/catch
107+
108+
If your `Logger` implementation throws — a cyclic-meta `JSON.stringify`, a transport that rejects synchronously, a typo in a bound `this` — the exception is swallowed so it never masks the real error that triggered the log call (for example, a provider SDK failure inside the chat stream). You won't see the log line, but the pipeline error still surfaces through thrown exceptions and `RUN_ERROR` chunks.
109+
110+
If you need to know when your own logger is failing, guard inside your implementation:
111+
112+
```typescript
113+
const logger: Logger = {
114+
debug: (msg, meta) => {
115+
try {
116+
pinoLogger.debug(meta, msg);
117+
} catch (err) {
118+
// surface to wherever you track infra errors
119+
process.stderr.write(`logger failed: ${String(err)}\n`);
120+
}
121+
},
122+
// ... info, warn, error
123+
};
124+
```
125+
126+
## Categories reference
127+
128+
| Category | Logs | Applies to |
129+
|----------|------|------------|
130+
| `request` | Outgoing call to a provider (model, message count, tool count) | All activities |
131+
| `provider` | Every raw chunk/frame received from a provider SDK | Streaming activities (chat, realtime) |
132+
| `output` | Every chunk or result yielded to the caller | All activities |
133+
| `middleware` | Inputs and outputs around every middleware hook | `chat()` only |
134+
| `tools` | Before/after tool call execution | `chat()` only |
135+
| `agentLoop` | Agent-loop iterations and phase transitions | `chat()` only |
136+
| `config` | Config transforms returned by middleware `onConfig` hooks | `chat()` only |
137+
| `errors` | Every caught error anywhere in the pipeline | All activities |
138+
139+
## Errors are always logged
140+
141+
Errors flow through the logger unconditionally — even when you omit `debug`:
142+
143+
```typescript
144+
chat({ adapter, messages }); // still prints [tanstack-ai:errors] ... on failure
145+
```
146+
147+
To fully silence (including errors), set `debug: false` or `debug: { errors: false }`. Errors also always reach the caller via thrown exceptions or `RUN_ERROR` stream chunks — the logger is additive, not the only surface.
148+
149+
## Non-chat activities
150+
151+
The same `debug` option works on every activity:
152+
153+
```typescript
154+
summarize({ adapter, text, debug: true });
155+
generateImage({ adapter, prompt: "a cat", debug: { logger } });
156+
generateSpeech({ adapter, text, debug: { request: true } });
157+
```
158+
159+
The chat-only categories (`middleware`, `tools`, `agentLoop`, `config`) simply never fire for these activities because those concepts don't exist in their pipelines.
160+
161+
## Related
162+
163+
If you're building middleware and want to see chunks flow through it, `debug: { middleware: true }` is faster than writing a logging middleware. See [Middleware](./middleware) for writing your own middleware, or [Observability](./observability) for the programmatic event client.

docs/advanced/extend-adapter.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
---
22
title: Extend Adapter
33
id: extend-adapter
4-
order: 7
4+
order: 8
55
description: "Extend TanStack AI adapter factories with custom model IDs and fine-tuned models while keeping full type safety for input modalities and provider options."
66
keywords:
77
- tanstack ai

docs/advanced/middleware.md

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -49,6 +49,9 @@ const stream = chat({
4949
});
5050
```
5151

52+
> **Just want to see chunks flowing through your middleware during development?**
53+
> Use `debug: { middleware: true }` on your `chat()` call — no custom middleware required. See [Debug Logging](./debug-logging).
54+
5255
## Lifecycle Overview
5356

5457
Every `chat()` invocation follows a predictable lifecycle. Middleware hooks fire at specific phases:

docs/advanced/multimodal-content.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
---
22
title: Multimodal Content
33
id: multimodal-content
4-
order: 3
4+
order: 4
55
description: "Send images, audio, video, and documents alongside text in TanStack AI messages with typed ContentPart primitives for multimodal models."
66
keywords:
77
- tanstack ai

docs/advanced/observability.md

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -19,6 +19,9 @@ The `@tanstack/ai` package offers you an event client for observability and debu
1919
It's a fully type-safe decoupled event-driven system that emits events whenever they are internally
2020
triggered and you can subscribe to those events for observability.
2121

22+
> **Looking for quick diagnostic console output instead of a programmatic event stream?**
23+
> See [Debug Logging](./debug-logging) for turning on category-toggleable logging across every adapter and middleware hook.
24+
2225
Because the same event client is used for both the TanStack Devtools system and observability locally it will work
2326
by subscribing to the event bus and emitting events to/from the event bus into the listeners by default. If you
2427
want to subscribe to events in production as well you need to pass in a third argument to the `on` function,

docs/advanced/per-model-type-safety.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
---
22
title: Per-Model Type Safety
33
id: per-model-type-safety
4-
order: 4
4+
order: 5
55
description: "TanStack AI narrows modelOptions and content types to the specific model you select, enforcing capabilities at compile time."
66
keywords:
77
- tanstack ai

docs/advanced/runtime-adapter-switching.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
---
22
title: Runtime Adapter Switching
33
id: runtime-adapter-switching
4-
order: 5
4+
order: 6
55
description: "Let users switch between LLM providers at runtime in TanStack AI while keeping full TypeScript type safety for each adapter's model options."
66
keywords:
77
- tanstack ai

docs/advanced/tree-shaking.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
---
22
title: Tree-Shaking
33
id: tree-shaking
4-
order: 6
4+
order: 7
55
description: "TanStack AI's tree-shakeable architecture — import only the activities and adapters you use for minimal bundle size across chat, image, and speech."
66
keywords:
77
- tanstack ai

0 commit comments

Comments
 (0)