fix: remove Anthropic structured outputs beta header (now GA)#4987
fix: remove Anthropic structured outputs beta header (now GA)#4987Wh1isper wants to merge 3 commits intopydantic:mainfrom
Conversation
33ff459 to
0f08b27
Compare
Co-Authored-By: YAAI <261597362+yaai-bot@users.noreply.github.com>
0f08b27 to
fb33281
Compare
Using this header works fine with the official Anthropic API, but I encountered this error when using an internal gateway to route requests from Anthropic to the Anthropic model on Vertex AI. This may be due to inconsistencies in how the header behaves between the official Anthropic API and Vertex AI. |
|
@Wh1isper Can we have a VCR update? |
|
I ran the full test suite against the existing cassettes and all 24 tests in I don't have an official Anthropic API key, but I can record via a gateway if needed. Let me know if you'd like me to add additional VCR coverage for the beta header scenarios specifically. |
|
In this case, I find it useful to record the cassette again. We want to confirm the behavior of the provider itself. The cassettes don't change because they don't care about headers matching. @adtyavrdhn Can you find a useful test that was using this functionality and record the cassette again, please? One is fine. |
Kludex
left a comment
There was a problem hiding this comment.
Good clean up here. 👍
Approving pending the cassette update.
| """Agent with no tools and no output_type.""" | ||
| model = anthropic_model('claude-sonnet-4-5') | ||
| hook = create_header_verification_hook(expect_beta=False, test_name='test_no_tools_no_output') | ||
| model.client._client.event_hooks['request'].append(hook) # pyright: ignore[reportPrivateUsage] | ||
|
|
||
| agent = Agent(model) | ||
| agent.run_sync('Tell me a brief fact about Paris') |
There was a problem hiding this comment.
🚩 Many VCR integration tests now have zero explicit assertions
After removing the create_header_verification_hook checks, 11 tests in tests/models/anthropic/test_output.py now have no assertions at all — they just call agent.run_sync() and return. For example, test_no_tools_no_output (line 214-217), test_strict_true_tool_no_output (line 278-286), test_mixed_tools_no_output (line 412-424), etc. The VCR cassettes only match on HTTP method and URL path (see custom matchers at tests/conftest.py:323-335), so they don't validate request body content. These tests now effectively only verify that the code doesn't crash. The tests/AGENTS.md rule:334 says to "assert meaningful behavior in tests, not just code execution." While removing the beta header assertions is correct (the feature is GA), these tests could benefit from either (a) being removed entirely if they no longer test distinct behavior, or (b) having new assertions added (e.g., verifying tool schemas, output_config presence, or snapshotting result.all_messages() per rule:194).
Was this helpful? React with 👍 or 👎 to provide feedback.
structured-outputs-2025-11-13beta header (now GA) #4988Anthropic structured outputs (native JSON schema output and strict tool calls) have graduated from beta to GA. The beta header is no longer required and the
output_configparameter is already used (the GA name).See: https://platform.claude.com/docs/en/build-with-claude/structured-outputs
Changes:
structured-outputs-2025-11-13beta header in_get_betas_and_extra_headers()toolsandmodel_request_parametersparametersPre-Review Checklist
make formatandmake typecheck.Pre-Merge Checklist