refactor(model-routing): centralize explicit foreground fallback policy - #6020
Open
alteixeira20 wants to merge 1 commit into
Open
refactor(model-routing): centralize explicit foreground fallback policy#6020alteixeira20 wants to merge 1 commit into
alteixeira20 wants to merge 1 commit into
Conversation
Make foreground fallback an explicit per-user, availability-only policy shared by streaming Chat, non-stream Chat, and Agent runs. Preserve strict defaults, owner/model and credential boundaries, pinned Agent routes, and truthful per-round provenance/accounting. Carry provider-reported model identifiers through native streaming adapters, non-stream responses, and caches, and keep legacy default_model_fallbacks as tombstoned raw storage that generic settings APIs and agent tools cannot expose or mutate.
| try { | ||
| const runCosts = JSON.parse(localStorage.getItem(_COST_RUNS_KEY) || '{}'); | ||
| const sessionRuns = runCosts[sid] && typeof runCosts[sid] === 'object' | ||
| ? runCosts[sid] |
| if (entries.length > _MAX_COST_RUNS_PER_SESSION) { | ||
| const overflow = entries.slice(0, entries.length - _MAX_COST_RUNS_PER_SESSION); | ||
| const costs = JSON.parse(localStorage.getItem(_COST_KEY) || '{}'); | ||
| costs[sid] = (costs[sid] || 0) + overflow.reduce( |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Centralize explicit foreground fallback policy for Chat and Agent routing.
Fallback remains disabled by default, advances only on eligible availability
failures, preserves owner/model/credential boundaries, pins the answering Agent
route after substantive output or a completed tool call, and records truthful
per-round model/endpoint provenance and usage attribution.
This is the clean rebased replacement for #5811 on top of merged #5801. The
implementation was reconciled against current
devand the rebase regressionsfound during final audit were fixed.
Target branch
dev, notmain.Linked Issue
Fixes #5626
Supersedes #5811.
Part of #5625.
Type of Change
Checklist
devdocker compose uporuvicorn app:app) and verified the change works end-to-end.Behavior contract
default_model_fallbacksremains stored but is not used as foreground fallback policy.Validation
Final rebase audit was performed against current
dev._ody_general_no_tool_modereference removed.How to Test
requirements.txtin a clean virtual environment.prefs, and LLM fallback tests.
python -m pytest -q.python -m compileall -q core routes src.node --checkagainst the modified/new Chat JavaScript modules.may advance to an explicitly configured route.
and saved metrics identify the actual answering route.
Visual / UI changes — REQUIRED if you touched anything that renders
This changes existing Chat provenance and terminal-stream behavior rather than
introducing a new visual component or layout. Existing visual styling is
preserved. Browser-side JavaScript regression tests cover provenance, terminal
errors, background terminal state, and interrupted streams.
No new visual surface is introduced.