Skip to content

fix: log Anthropic system message first in span input#2109

Open
Joshua Tjhin (joshua-tj) wants to merge 1 commit into
braintrustdata:mainfrom
joshua-tj:fix/anthropic-system-message-first
Open

fix: log Anthropic system message first in span input#2109
Joshua Tjhin (joshua-tj) wants to merge 1 commit into
braintrustdata:mainfrom
joshua-tj:fix/anthropic-system-message-first

Conversation

@joshua-tj

Copy link
Copy Markdown

Problem

The Anthropic instrumentation merges the top-level system param into the logged input messages array (since the Anthropic API takes it separately rather than as a message). But coalesceInput appends it after the conversation messages, so every anthropic.messages.create span logs:

[
  { "role": "user", "content": "..." },
  { "role": "system", "content": "..." }
]

This doesn't match what the model actually receives (the API applies the system prompt first), and it makes logged calls awkward to reuse: opening a logged span in the playground requires manually reordering the messages before rerunning, since chat-message convention puts system first.

Fix

coalesceInput now prepends the system message instead of appending it. Logged inputs read [system, user, ...].

  • Exported coalesceInput and added unit tests (system-first ordering, system as text-block array, no system prompt, no mutation of the caller's array) — it previously had no direct coverage.
  • Regenerated the anthropic-instrumentation e2e span-tree snapshots via test:e2e:update; the diffs are purely the system message moving from last to first.

Verification

  • js/ core suite: 1070 passed
  • anthropic-instrumentation e2e scenario run twice: 184 passed both times
  • tsc --noEmit and prettier clean

Note: this changes the logged input shape, so anything downstream that assumes the system message is last (e.g. saved playground sessions or scorers parsing input) will see the new order. The in-repo e2e assertions already look up the system message by role rather than position, so none needed changes.

🤖 Generated with Claude Code

The Anthropic instrumentation merges the top-level `system` param into
the logged messages array, but appended it after the conversation
messages. The model actually receives the system prompt first, and
downstream consumers (e.g. opening a logged span in the playground)
expect chat-message order, so a trailing system message had to be
manually reordered before reuse.

Prepend the system message instead, and update the e2e span-tree
snapshots to match.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant