Add Foundry Local provider#330
Open
justinchuby wants to merge 9 commits into
Open
Conversation
In explicit-endpoint mode the model string was passed through unchanged, so users had to pass the concrete served id (e.g. qwen2.5-0.5b-instruct-generic-cpu:4) discovered from the endpoint, while managed mode accepted a friendly alias. Resolve the alias against the endpoint's /v1/models (exact id, then parent alias, then alias-boundary prefix), falling back to the original string when the lookup fails or no confident match is found. This lets the same foundry_local:<alias> work in both modes. Also document the aisuite[mcp] requirement for tool calling. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Adds a first-class `foundry_local` provider to the coworker platform, mirroring the existing on-device Ollama integration: - registry: `foundry_local` descriptor (keyless, optional user-supplied `base_url` since the Foundry Local port is dynamic) built on the shared OpenAI-compatible client; generalize `_normalize_ollama_url` into `_normalize_openai_compatible_url`. - capabilities: treat foundry_local like ollama (tools on, single tool call, no vision, streaming). - manager: `_foundry_models` lists models live from the endpoint's `/v1/models` for the settings model picker. Verified end-to-end against a running Foundry Local service (qwen2.5-0.5b): provider builds, routes, completes, and reports capabilities; model listing returns the served id. Full platform suite (332 passed, 1 skipped). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The model-config UI hard-coded Ollama as the only local provider: the 'Local models' pane targeted `ollama` and the 'API models' pane listed every provider except `ollama`. A keyless provider like `foundry_local` therefore fell into the API pane, which only renders an `api_key` field — so its `base_url` couldn't be set at all. Generalize both the onboarding and Manage-models panes to treat any keyless provider (`needs_key == false`) as local: - The Local pane now has a runtime selector (shown when more than one local runtime exists) and renders the `base_url` field (label/placeholder/help) from the selected provider's descriptor, so Ollama and Foundry Local each get correct copy. - The API pane lists `needs_key` providers instead of 'everything but ollama'. No backend changes; the panes are now driven by provider metadata. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds support for Microsoft Foundry Local, an on-device runtime that exposes an OpenAI-compatible API.
FoundryLocalProvider(named after the product) subclassesOpenaiProvider(same pattern asollama/lmstudio) and is selected with thefoundry_local:model prefix. AFoundry_localProvideralias is kept so the provider factory (which derives the class name from"foundry_local".capitalize()) resolves the key. The provider operates in one of two modes:foundry-local-sdk1.x (imported asfoundry_local_sdk, using theConfiguration/catalog/start_web_servicesingleton API) and the legacy 0.x package (imported asfoundry_local, usingFoundryLocalManager(alias)).api_url/base_urlorFOUNDRY_LOCAL_API_URLis set, points the OpenAI SDK directly at a running endpoint (no SDK required); the host is normalized to/v1idempotently. A friendly model alias is resolved to the concrete served id by querying the endpoint's/v1/models(exact id →parentalias → alias-boundary prefix), falling back to the original string when the lookup fails or there is no confident match — so the samefoundry_local:<alias>works in both modes.Validation
Tested end-to-end on Windows against a real Foundry Local install (
0.8.119) withfoundry-local-sdk 1.2.3:foundry_local:qwen2.5-0.5b): auto-started the service, downloaded + loaded the model, resolved the alias toqwen2.5-0.5b-instruct-generic-gpu:4, returned a valid completion (finish_reason=stop).qwen2.5-0.5bresolved against the running endpoint toqwen2.5-0.5b-instruct-generic-cpu:4and returned a completion.tool_callsresponse (finish_reason=tool_calls). Note: tool calling throughai.Clientcurrently requires themcpextra (pip install 'aisuite[mcp]') due to an unrelated upstream bug whereis_mcp_configis gated behind the MCP import.Changes
aisuite/providers/foundry_local_provider.py— new provider (FoundryLocalProvider, with aFoundry_localProvideralias for factory compatibility); explicit-endpoint alias resolution.pyproject.toml—foundry-localextra installsopenaiandfoundry-local-sdk(^1.2.3, declared with apython = ">=3.11"marker since the SDK requires Python ≥3.11 while aisuite supports^3.10; it's also added to theallextra). The SDK pulls in native ONNX Runtime/CUDA wheels, so users on Python 3.10 (or who need a hardware-specific variant) can still install it directly withpip install foundry-local-sdk.tests/providers/test_foundry_local_provider.py— explicit-endpoint mode (incl. alias resolution by prefix/parent, concrete-id passthrough, ambiguous-alias error, unreachable-endpoint fallback), both managed SDK backends (1.x and 0.x), alias→id resolution, multi-alias loading, missing-SDK error.guides/foundry_local.md(+ index link inguides/README.md) — usage for both modes and theaisuite[mcp]note for tool calling.Usage
Explicit endpoint (SDK not required; alias resolved against the running endpoint):