Skip to content

Commit d43a3df

Browse files
authored
Warn about small context windows on self-hosted AI providers (#7776)
Replaces #7770, which put this in `engine/.env` where an admin configuring AI through the settings UI would never see it. Shown only when the provider is Ollama or Custom (OpenAI-compatible), directly under the existing SSRF warning on the base URL field - the point where someone is actually pointing the engine at their own endpoint. Ollama defaults `OLLAMA_CONTEXT_LENGTH` to 4096 and divides it further between parallel request slots, leaving roughly 2,000 tokens of usable prompt. Measured against a local qwen3:8b: prompts up to ~2,685 tokens arrive intact, anything larger is clipped to exactly 2,050, and the overflow is discarded from the **start** of the prompt - which is where the planner puts the user's request. The model then selects tools without having seen what was asked. Nothing errors and nothing logs. The engine cannot set this itself: it is an Ollama server option with no equivalent on the OpenAI-compatible endpoint the engine speaks. So the useful thing it can do is tell the person configuring it, at the moment they configure it. String added to en-GB and en-US only; other locales pick it up through the usual translation flow and fall back to English until then.
1 parent 84ac22a commit d43a3df

2 files changed

Lines changed: 18 additions & 0 deletions

File tree

frontend/editor/public/locales/en-US/translation.toml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -818,6 +818,7 @@ openai = "OpenAI API key"
818818
setPlaceholder = "Saved - leave blank to keep the current key"
819819

820820
[admin.settings.ai.models.baseUrl]
821+
contextWindow = "Use a context window of at least 16,384 tokens. Smaller windows silently drop part of the prompt. Ollama defaults to 4,096 - raise OLLAMA_CONTEXT_LENGTH."
821822
description = "Base URL of the OpenAI-compatible / Ollama endpoint, e.g. http://ollama:11434/v1."
822823
label = "Provider base URL"
823824
warning = "The base URL must point at a trusted internal endpoint. The engine will make server-side requests to it, so an untrusted value is SSRF-sensitive."

frontend/editor/src/proprietary/components/shared/config/configSections/AdminAiModelsSection.tsx

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -370,6 +370,23 @@ export default function AdminAiModelsSection() {
370370
</Text>
371371
</Alert>
372372
)}
373+
374+
{showBaseUrl && (
375+
<Alert
376+
variant="light"
377+
color="blue"
378+
icon={
379+
<LocalIcon icon="info-rounded" width="1rem" height="1rem" />
380+
}
381+
>
382+
<Text size="xs">
383+
{t(
384+
"admin.settings.ai.models.baseUrl.contextWindow",
385+
"Use a context window of at least 16,384 tokens. Smaller windows silently drop part of the prompt. Ollama defaults to 4,096 - raise OLLAMA_CONTEXT_LENGTH.",
386+
)}
387+
</Text>
388+
</Alert>
389+
)}
373390
</Stack>
374391
</Paper>
375392
</Stack>

0 commit comments

Comments
 (0)