Make LLM backend any OpenAI-compatible endpoint#11
Open
ethenotethan wants to merge 1 commit intomainfrom
Open
Conversation
Drops the hardcoded OpenRouter base URL and Claude Agent SDK framing in favor of generic OpenAI Chat Completions config (OPENAI_API_KEY, OPENAI_BASE_URL, OPENAI_MODEL), so Flashlight works against OpenAI, OpenRouter, vLLM, LM Studio, Ollama, and anything else that speaks the same protocol + tool-calling schema. - agent/burr_app.py: parametrize base URL, rename client helper to _chat_completion (avoid collision with the Burr call_llm action), only emit OpenRouter attribution headers when routing through openrouter.ai, default model -> gpt-4o-mini - agent/cli.py, agent/agent.py: swap API-key checks to OPENAI_API_KEY with provider-agnostic error messages - .env.example: document OPENAI_* vars + per-provider examples - README.md: drop Claude Agent SDK references and add an LLM configuration section with an env-var table and example configs
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
OPENAI_API_KEY(required),OPENAI_BASE_URL(defaulthttps://api.openai.com/v1),OPENAI_MODEL(defaultgpt-4o-mini). NoOPENROUTER_*backwards compat.ANTHROPIC_API_KEY; adds an LLM configuration section with an env-var table and per-provider example configs.Changes
agent/burr_app.py— parametrized base URL viaget_base_url(), renamed client helpercall_openrouter→_chat_completion(avoids collision with the existing Burrcall_llmaction), OpenRouter-specificHTTP-Referer/X-Titleattribution headers now only emitted when the base URL actually containsopenrouter.ai.agent/cli.py,agent/agent.py— API-key checks swapped toOPENAI_API_KEYwith provider-agnostic error messages..env.example— rewritten withOPENAI_*vars and example configs for OpenAI / OpenRouter / local vLLM / LM Studio / Ollama.README.md— intro + Installation + Requirements + Contributing sections de-Claude'd; de-emoji'd verbose-logging bullets.Compatibility constraint
"OpenAI-compatible" here specifically means the server returns structured
tool_callson the response message (not JSON emitted as text incontent). Flashlight's ReAct loop relies onresponse["tool_calls"]in_chat_completion; an endpoint that only emits tool-call intent as plain text will stall the loop after one turn. Works well with OpenAI, OpenRouter (Claude/GPT/Gemini), vLLM, LM Studio,mlx-omni-server, and other servers that implement the OpenAI tools schema.Test plan
python -m compileall agent/python -c "from agent.burr_app import get_api_key; get_api_key()"