Would an OrcaRouter integration be welcome? (similar shape to aimlapi/cerebras)

[jinhaosong-source]

Hi all,

I'm exploring whether an OrcaRouter integration would be a welcome addition
to Haystack. Before doing any implementation work, I wanted to check with
maintainers and the community first.

What OrcaRouter does

It's an OpenAI-compatible LLM gateway that routes requests across multiple
model providers (Anthropic, OpenAI, Google, Mistral, Aleph Alpha, and others)
through a single endpoint, with adaptive cost/latency-based provider selection.
Conceptually it sits in the same category as the existing aimlapi and
cerebras integrations.

Why I'm asking before opening a PR

Haystack already supports similar multi-provider gateways (aimlapi, cerebras),
plus a generic OpenAI-compatible path via the base OpenAI generator. So before
doing the work, I'd like to understand:

1. Is there appetite for another integration in this category, or is the
   preferred path now "use the generic OpenAI generator with a custom
   api_base_url"?
2. If a dedicated integration is welcome, would the aimlapi structure
   (subclassing the OpenAI generator with provider-specific defaults and tests)
   be the right shape to follow?

Possible value for Haystack users

The angle that might be most relevant, given deepset's positioning: OrcaRouter
provides EU-resident inference paths (Frankfurt, Paris) and native coverage of
Mistral, Aleph Alpha, and Pleias alongside the standard providers. For
Haystack users building under GDPR / EU AI Act constraints, a provider option
that keeps traffic in-region without giving up frontier-model access might be
useful — but I'd rather hear from the community whether this is a real demand
signal before assuming it is.

Happy to hear honest feedback, including "the generic adapter is enough."

Thanks!

[reallyticsai]

We've built production-grade AI systems with similar multi-provider gateways, such as our voice agents with sub-300ms latency. An OrcaRouter integration could be valuable for Haystack users, particularly those under GDPR and EU AI Act constraints, as it provides EU-resident inference paths and native coverage of various providers.
To implement this integration, subclassing the OpenAI generator with provider-specific defaults and tests, similar to the aimlapi structure, seems like a suitable approach.
We recommend exploring the existing aimlapi and cerebras integrations as references.
You can also consider using a configuration-driven approach to handle different providers and their respective settings.
For example, you could define a configuration schema that includes provider-specific parameters, such as api_base_url and model_name, and then use this configuration to initialize the OrcaRouter integration.
Here's an example of how you might structure this configuration:

orcarouter_config = {
    "api_base_url": "https://api.orcarouter.com/v1",
    "model_name": "mistral-7b",
    "provider": "mistral"
}

[julian-risch]

We have a follow up issue now https://github.com/deepset-ai/haystack/issues/11591