Skip to content

LLMService.Generate does not support Anthropic-compatible endpoints using v1/messages #14

@helei-ai

Description

@helei-ai

Describe the bug

LLMService.Generate currently assumes an OpenAI-style upstream path and payload shape. As a result, Anthropic-compatible gateways may fail even when the gateway itself is valid and reachable.

To Reproduce

Set the following environment variables:

LLM_API_ENDPOINT=https://<gateway>/api/anthropic
LLM_API_KEY=...
LLM_MODEL=feature/gpt

Then call the server-side LLM generation path (for example through LLMService.Generate).

Expected behavior

When LLM_API_ENDPOINT points to an Anthropic-compatible gateway, the server-side LLM client should be able to:

  • call v1/messages
  • send Anthropic-compatible headers
  • send an Anthropic-compatible request body
  • parse the Anthropic-compatible response

Actual behavior

The current implementation normalizes requests as OpenAI-style requests, which causes Anthropic-compatible endpoints to fail.

Additional context

The project already exposes generic environment variables:

  • LLM_API_ENDPOINT
  • LLM_API_KEY
  • LLM_MODEL

However, the current client behavior is not protocol-aware enough to support Anthropic-compatible gateways behind those generic settings.

A protocol-aware implementation would likely need:

  • OpenAI-style handling via /v1/responses
  • Anthropic-style handling via /v1/messages

with the corresponding headers, payload shape, and response parsing logic.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions