Skip to content

enable model discovery for local Ollama BYOK providers - #6079

Draft
dpeterson01 wants to merge 1 commit into
nexu-io:mainfrom
dpeterson01:feat/ollama-model-discovery
Draft

enable model discovery for local Ollama BYOK providers#6079
dpeterson01 wants to merge 1 commit into
nexu-io:mainfrom
dpeterson01:feat/ollama-model-discovery

Conversation

@dpeterson01

@dpeterson01 dpeterson01 commented Jul 25, 2026

Copy link
Copy Markdown

Fixes #6077

Why

I hit this running Open Design against a local Ollama server through BYOK: every other provider protocol can pull its catalogue with "Fetch models," but Ollama couldn't, even though Ollama exposes a discovery endpoint (GET /api/tags). The button was disabled and the daemon returned unsupported_ollama, so I had to type each model id by hand.

Per the discussion in #6077 with @lefarcen, this keeps the scope to local Ollama only (loopback base URLs). Ollama Cloud (ollama.com) keeps its existing "not available" fallback. Remote/LAN and host.docker.internal are left as a separate future expansion.

What users will see

When the Ollama Base URL is a loopback host (localhost / 127.0.0.1 / ::1), the Fetch models button is now enabled and populates the model picker from the running Ollama's installed models. No API key is required for local Ollama. Nothing changes for Ollama Cloud, which still shows the existing unsupported message. The behavior is consistent across the surfaces that offer model discovery: onboarding BYOK, Settings, the inline model switcher, and the avatar menu.

Surface area

  • UI — the Fetch models control is enabled for local Ollama in onboarding, Settings, the inline switcher, and the avatar menu
  • API / contract — no new endpoint; reuses POST /api/provider/models. The daemon now supports the ollama protocol there (loopback-gated) and no longer requires an API key for local Ollama discovery

Screenshots

Onboarding → Bring your own key → Ollama, local base URL. After Fetch models, the picker lists the installed models pulled from /api/tags:
open-design-ollama-discovery

Bug fix verification

Not a bug fix, but the change is covered by falsifiable tests:

  • apps/daemon/tests/connection-test.test.ts: local Ollama lists models from /api/tags; Ollama Cloud is rejected as unsupported without calling upstream fetch.
  • apps/web/tests/components/SettingsDialog.test.ts: canFetchProviderModels / isProviderModelDiscoveryUnsupported return the expected results for loopback vs cloud Ollama.

Validation

  • pnpm --filter @open-design/daemon exec vitest run tests/connection-test.test.ts: 158/158
  • pnpm --filter @open-design/web test: 4762 passed, 7 skipped
  • pnpm guard: pass
  • pnpm typecheck (root, all workspaces): pass
  • Built the production Docker image and manually verified against a real local Ollama: Fetch models returned the installed catalogue and the picker populated.

Implementation notes

  • Daemon (apps/daemon/src/integrations/provider-models.ts): added an Ollama /api/tags URL, a response parser, and a loopback-only guard so cloud endpoints keep the existing unsupported result.
  • Route (apps/daemon/src/routes/chat.ts): POST /api/provider/models required an API key for every protocol except AIHubMix/Bedrock; local Ollama is now also exempt since /api/tags is unauthenticated.
  • Frontend gates reuse the existing isLocalOllamaBaseUrl helper (apps/web/src/utils/byokProvider.ts), so the loopback definition stays in one place.

@lefarcen

Copy link
Copy Markdown
Contributor

Hey @dpeterson01 — keeping Ollama discovery loopback-only while reusing the existing fetch-model flow is a nice, well-scoped direction for this PR.

Since this touches user-facing model selection, I’ve marked it for a manual QA pass before merge; we’ll loop that in once it’s merge-ready.

@lefarcen
lefarcen requested a review from PerishCode July 25, 2026 20:30
@lefarcen lefarcen added size/M PR changes 100-300 lines risk/high High risk: apps/desktop, daemon, auth, migration, workflows, package deps type/feature New feature needs-validation Runtime change detected; needs human or /explore agent validation. labels Jul 25, 2026
@dpeterson01
dpeterson01 force-pushed the feat/ollama-model-discovery branch from 9d9de2a to f959caa Compare July 28, 2026 19:39

@PerishCode PerishCode left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@dpeterson01 I reviewed the complete changed ranges across the daemon discovery path and all four web model-selection entry points. The implementation keeps discovery loopback-only, preserves the existing SSRF validation boundary, avoids requiring credentials for local /api/tags, rejects cloud Ollama before upstream egress, and includes focused coverage for the daemon response plus the web support gates. The state and cache-key wiring remain consistent with the existing provider-model flow. Nice, well-contained addition—thanks for carrying the behavior through each user-facing picker.

🔁 Powered by Looper · runner=reviewer · agent=codex · An autonomous AI dev team for your GitHub repos.

@dpeterson01

Copy link
Copy Markdown
Author

CI block tracked in #6289 — the control-profile required checks are cancelling/stalling on the self-hosted runner pool, not anything specific to this PR.

@lefarcen

Copy link
Copy Markdown
Contributor

Thanks for the pointer — understood. We'll treat the current block as the runner-pool issue tracked in #6289 rather than anything specific to this diff, and keep this PR parked until the control-profile required checks are healthy again.

@lefarcen lefarcen added the needs-product-review Feature PR awaiting product sign-off before merge (see roadmap) label Jul 30, 2026
@lefarcen
lefarcen marked this pull request as draft August 2, 2026 19:17
Wire up daemon provider-model discovery for the ollama protocol (GET /api/tags
+ response parser), gated to local/loopback base URLs so Ollama Cloud keeps its
existing unsupported fallback. Remove the frontend short-circuits that blocked
Fetch models for local Ollama across onboarding, settings, the inline switcher,
and the avatar menu, reusing the existing isLocalOllamaBaseUrl helper. Local
Ollama needs no API key. Adds daemon and web tests.
@dpeterson01
dpeterson01 force-pushed the feat/ollama-model-discovery branch from f959caa to ba4e978 Compare August 8, 2026 06:41
@dpeterson01

Copy link
Copy Markdown
Author

Rebased onto current main to clear the merge conflict this had picked up. The only conflict was in apps/web/src/components/AvatarMenu.tsx: main refactored the model-fetch effect (added a URL-validity check and reworked the "AIHubMix is public, others need a key" gate). I resolved it to keep that refactor and re-apply the local-Ollama allowance, so Azure and cloud Ollama stay blocked while local (loopback) Ollama is allowed and skips the key requirement. The other seven files auto-merged.

One transparency note: I couldn't run the test suite for this rebase. My build box is offline and my current network is blocking the npm registry, so pnpm install isn't possible from here right now. The behavior is unchanged from the approved version; the rebase only reconciles it with main's AvatarMenu refactor. Could you re-run CI or take another quick look on your side? Happy to push fixups if anything turns up.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

needs-product-review Feature PR awaiting product sign-off before merge (see roadmap) needs-validation Runtime change detected; needs human or /explore agent validation. risk/high High risk: apps/desktop, daemon, auth, migration, workflows, package deps size/M PR changes 100-300 lines type/feature New feature

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Enable model discovery (Fetch models) for the Ollama BYOK provider

3 participants