Read images with deepseek-v4-flash-vision-exp, and move DeepSeek onto the Responses protocol

[hiyouga]

deepseek-v4-flash-vision-exp reads images, DeepSeek's dedicated client moves off Chat
Completions onto the Responses API, and a replay-ordering bug that both Responses clients
carried is fixed.

What the live API does

Probed against api.deepseek.com on 2026-08-21 with a picture whose only content is the word
BANANA, never named in the prompt:

Probe	Result
Image in a user message (data URL and HTTP URL)	reads BANANA on both protocols
Image in a tool result (function_call_output with input_image)	reads BANANA
Same turn, tool result without the image (control)	answers NO_IMAGE, so the image really is what it read
Image sent to a non-vision model	Chat Completions: 400 This model does not support image. Responses: no error, the image becomes placeholder text and the model answers "Unsupported"
reasoning.effort: "none"	0 reasoning tokens — this is what disables thinking
extra_body {"thinking": {"type": "disabled"}}	ignored on /responses: still 148 reasoning tokens
tool_choice: "required" or a named tool	works only with thinking off; otherwise 400 Thinking mode does not support this tool_choice
temperature 0.0 and 2.0	accepted (documented as having no effect in thinking mode)
service_tier: "priority"	silently ignored
api.siliconflow.cn/v1/responses	404 page not found
openrouter.ai/api/v1/responses with deepseek/deepseek-v4-flash	works

Replay probes on the captured tool-call turn: the reasoning item can come back as
reasoning_text content, as a summary, empty, with encrypted_content, or not at all — every
shape is accepted with behavior intact, so the client rebuilds the minimal one from the
thinking text.

The ordering bug

test_tool_use failed intermittently against DeepSeek with
400 No tool output found for tool call <id>. It happens whenever the assistant speaks before
calling a tool: the transform appended the assistant message after the function_call it
preceded, and DeepSeek merges a call into the adjacent assistant message. Isolated probe:

• user → reasoning → function_call → assistant text → function_call_output → 400
• user → reasoning → assistant text → function_call → function_call_output → works

Both DeepSeekV4Client and OpenaiResponsesClient now flush collected message text before
appending a standalone input item. The generic client carried the same latent bug.

Decisions taken (confirmed in the conversation)

• SiliconFlow has no /responses, so its two DeepSeek entries move to the generic
  openai-chat client rather than keeping a second DeepSeek client alive.
• thinking_level is pre-mapped to what DeepSeek settles on (none/low/high/high/high/max)
  instead of passing medium/xhigh through.
• temperature and tool_choice keep the current strict behavior: UnsupportedParameterError
  for a temperature other than 1.0 and for anything but auto/none.
• Images on a non-vision model raise instead of being forwarded, because the Responses API
  degrades silently rather than failing.
• Unit tests: the DeepSeek cases in the four protocol suites move to the Responses wire
  shape; the empty-response and tool-call-argument suites also gained a case for the generic
  openai-responses client, which had none.

Also in this change

README.md gained a table naming the wire protocol behind every client_type: google-genai
for the Gemini family, ant-messages for Claude, openai-responses for GPT, DeepSeek and
MiniMax, openai-chat for GLM and Kimi, and the OpenAI Embeddings endpoint for
openai-embedding.

Verification

• E2E: pytest -k deepseek 35 passed (image URL, base64 image and tool-result image among
  them), jest -t deepseek 77 passed.
• pytest -k openai-responses 9 passed across gateways, covering the shared ordering fix.
• Offline suites: 196 passed (Python) and 184 passed (TypeScript); ruff check /
  ruff format --check, npm run lint and npm run build clean.
• Registry entries still construct through AutoLLMClient (test_list_supported_models).

Not covered

deepseek-v4-flash-vision-exp is an experimental model with no entry in the vendor's changelog;
if it is withdrawn, the registry entry and the official E2E slot go with it.

🤖 Generated with Claude Code

https://claude.ai/code/session_01UBwEdjegVGtfY2yJKwhZ77

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.