Skip to content

Feat/add litellm provider#168

Open
RheagalFire wants to merge 4 commits into
u14app:mainfrom
RheagalFire:feat/add-litellm-provider
Open

Feat/add litellm provider#168
RheagalFire wants to merge 4 commits into
u14app:mainfrom
RheagalFire:feat/add-litellm-provider

Conversation

@RheagalFire

Copy link
Copy Markdown

Summary

  • Adds LiteLLM as a first-class provider type in Neo Chat, giving users access to 100+ LLM providers (AWS Bedrock, Azure, Vertex AI, Cohere, Replicate, etc.) through a single proxy endpoint.
  • Includes proper base URL resolution, outbound security validation, and comprehensive tests.
  • Fixes a pre-existing bug where createOpenAIClient() hardcoded "OpenAI" for base URL resolution instead of using the actual provider type.

Test Plan

  • pnpm format:check
  • pnpm lint
  • pnpm typecheck
  • pnpm test (850 passed, 171 suites)
  • pnpm build

Notes

  • I updated docs or examples when behavior changed.
  • I considered local-first data handling and hosted deployment safety.
  • I added or updated tests for user-facing or security-sensitive changes.
  • For localization changes, I followed docs/localization-pr-guide.md and documented intentional English fallbacks.\

Changes

Provider registration (7 files):

  • src/lib/providers/types.ts - Added "LiteLLM" to ProviderType union
  • src/lib/providers/providerTypes.ts - Added LITELLM_PROVIDER_TYPE constant, updated isProviderType() and isOpenAIProviderType()
  • src/lib/api/schemas.ts - Added "LiteLLM" to the Zod runtime validation enum
  • src/lib/security/urlPolicy.ts - Added default LiteLLM base URL (http://localhost:4000/v1)
  • src/utils/apiHelpers.ts - Added "LiteLLM" to the ProviderConfig type union
  • src/components/settings/ProviderSettings.tsx - Added LiteLLM to provider type dropdown

Tests (4 files, +100 lines):

  • src/__tests__/providerRuntimeSchema.test.ts - LiteLLM schema validation
  • src/__tests__/urlPolicy.test.ts - 5 tests: default URL, custom URL with/without /v1, models path
  • src/__tests__/providerConfig.test.ts - 1 test: normalizeModelProvider preserves LiteLLM type
  • src/__tests__/providerOutbound.test.ts - 3 tests: client creation, localhost blocked in hosted mode, public URL allowed

Example usage

// In your app configuration or via the Settings UI:
const provider = {
  type: "LiteLLM" as const,
  baseUrl: "http://localhost:4000/v1",  // default, auto-filled in UI
  apiKey: "sk-your-litellm-key",        // or leave blank if no auth
};

// Models are auto-discovered from the proxy's /v1/models endpoint.
// Any model your LiteLLM proxy serves becomes available:
//   anthropic/claude-sonnet-4-6
//   bedrock/anthropic.claude-v2
//   vertex_ai/gemini-pro

Risk / Compatibility

  • Additive only. No existing provider types or streaming logic modified.
  • No new dependencies. LiteLLM proxy speaks the OpenAI protocol.
  • Hosted deployment safe. LiteLLM's default localhost:4000 is correctly blocked in hosted mode (tested).
  • Base URL fix benefits all providers. The provider.type fix is a correctness improvement for any future provider type.

@Amery2010

Amery2010 commented Jul 11, 2026

Copy link
Copy Markdown
Collaborator

Thank you for your support of this project.

The project currently supports OpenAI-compatible API formats, which are already a common format and support local models such as Ollama. Personally, I don't think it's necessary to add a general-purpose SDK, as this would make it difficult for users to choose the appropriate API type.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants