Skip to content

Generalize the external-provider interface and add AWS Bedrock as the first backend #178

Description

@ns-mkusper

Problem

The pack attaches external providers only over OpenAI Chat Completions + a bearer key, leaving most non-OpenAI backends unreachable — including Bedrock's Claude, Nova, and Llama, which Bedrock serves only over Converse.1 The bundled Envoy AI Gateway already implements the wire-schema translation and provider-specific auth (SigV4, Azure, GCP, workload identity) for these backends;23 the operator just never emits anything but schema: OpenAI + bearer. This is exposing existing capability, not building it.

Scope

  • Make the provider's wire schema, addressing, and credential independent, explicitly selected axes instead of assumed OpenAI + hostname + bearer key.
  • Keep existing OpenAI-compatible providers working unchanged.
  • Add AWS Bedrock as the first backend: Converse models on both endpoints under existing group access, authenticated by workload identity (no static key).
  • Make further backends (Azure, GCP Vertex, Anthropic direct, …) additive.2

How it's built is the implementer's call.

Acceptance criteria

  • A provider resource selects its backend explicitly, independent of credential type.
  • Existing OpenAI-compatible providers keep working unchanged.
  • A Bedrock provider serves Claude, Nova, and Llama through both endpoints under existing group access.
  • A backend authenticates with no committed static credential where workload identity exists.
  • Adding a backend needs a new variant only, not a change to the shared spec.
  • The interface and one non-OpenAI example are documented.

Open questions

  • Migrate the spec vs. keep the flat OpenAI fields alongside a typed block.
  • Whether the resource name still fits a general provider abstraction.

Footnotes

  1. AWS, API compatibility by modelshttps://docs.aws.amazon.com/bedrock/latest/userguide/models-api-compatibility.html : Chat Completions is checked only for OpenAI models; every Claude, Nova, and Llama row is Converse-only. Verified in us-west-2 (2026-08-26): the OpenAI endpoint returns model_not_found for us.anthropic.* / us.amazon.nova-* / us.meta.llama*, and succeeds only for openai.gpt-oss-*.

  2. Envoy AI Gateway, Supported Providershttps://aigateway.envoyproxy.io/docs/capabilities/llm-integrations/supported-providers/ : native-schema backends (OpenAI, Anthropic, AWS Bedrock, Azure OpenAI, GCP Vertex AI, GCP Anthropic, Cohere) plus OpenAI-compatible providers. 2

  3. Envoy AI Gateway, Connecting to AI Providershttps://aigateway.envoyproxy.io/docs/0.4/capabilities/llm-integrations/connect-providers/ : AWS credentials use the default chain including EKS Pod Identity and IRSA; specifying only a region uses workload identity with no static key.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    Priority

    None yet

    Start date

    None yet

    Target date

    None yet

    Size

    None yet

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions