Describe the bug
LLMService.Generate currently assumes an OpenAI-style upstream path and payload shape. As a result, Anthropic-compatible gateways may fail even when the gateway itself is valid and reachable.
To Reproduce
Set the following environment variables:
LLM_API_ENDPOINT=https://<gateway>/api/anthropic
LLM_API_KEY=...
LLM_MODEL=feature/gpt
Then call the server-side LLM generation path (for example through LLMService.Generate).
Expected behavior
When LLM_API_ENDPOINT points to an Anthropic-compatible gateway, the server-side LLM client should be able to:
- call
v1/messages
- send Anthropic-compatible headers
- send an Anthropic-compatible request body
- parse the Anthropic-compatible response
Actual behavior
The current implementation normalizes requests as OpenAI-style requests, which causes Anthropic-compatible endpoints to fail.
Additional context
The project already exposes generic environment variables:
LLM_API_ENDPOINT
LLM_API_KEY
LLM_MODEL
However, the current client behavior is not protocol-aware enough to support Anthropic-compatible gateways behind those generic settings.
A protocol-aware implementation would likely need:
- OpenAI-style handling via
/v1/responses
- Anthropic-style handling via
/v1/messages
with the corresponding headers, payload shape, and response parsing logic.
Describe the bug
LLMService.Generatecurrently assumes an OpenAI-style upstream path and payload shape. As a result, Anthropic-compatible gateways may fail even when the gateway itself is valid and reachable.To Reproduce
Set the following environment variables:
Then call the server-side LLM generation path (for example through
LLMService.Generate).Expected behavior
When
LLM_API_ENDPOINTpoints to an Anthropic-compatible gateway, the server-side LLM client should be able to:v1/messagesActual behavior
The current implementation normalizes requests as OpenAI-style requests, which causes Anthropic-compatible endpoints to fail.
Additional context
The project already exposes generic environment variables:
LLM_API_ENDPOINTLLM_API_KEYLLM_MODELHowever, the current client behavior is not protocol-aware enough to support Anthropic-compatible gateways behind those generic settings.
A protocol-aware implementation would likely need:
/v1/responses/v1/messageswith the corresponding headers, payload shape, and response parsing logic.