Skip to content

refactor(model-routing): centralize explicit foreground fallback policy - #6020

Open
alteixeira20 wants to merge 1 commit into
odysseus-dev:devfrom
alteixeira20:fix/rebase-pr-5811
Open

refactor(model-routing): centralize explicit foreground fallback policy#6020
alteixeira20 wants to merge 1 commit into
odysseus-dev:devfrom
alteixeira20:fix/rebase-pr-5811

Conversation

@alteixeira20

Copy link
Copy Markdown
Collaborator

Summary

Centralize explicit foreground fallback policy for Chat and Agent routing.

Fallback remains disabled by default, advances only on eligible availability
failures, preserves owner/model/credential boundaries, pins the answering Agent
route after substantive output or a completed tool call, and records truthful
per-round model/endpoint provenance and usage attribution.

This is the clean rebased replacement for #5811 on top of merged #5801. The
implementation was reconciled against current dev and the rebase regressions
found during final audit were fixed.

Target branch

  • This PR targets dev, not main.

Linked Issue

Fixes #5626

Supersedes #5811.

Part of #5625.

Type of Change

  • Bug fix
  • New feature
  • Refactor / cleanup

Checklist

  • I searched open issues and open PRs — this is the rebased replacement for refactor(model-routing): centralize explicit foreground fallback policy #5811, not a competing proposal.
  • This PR targets dev
  • My changes are limited to the explicit foreground fallback policy and its routing/provenance/test surface.
  • I actually ran the app (docker compose up or uvicorn app:app) and verified the change works end-to-end.

Behavior contract

  • Missing, disabled, malformed, or empty policy state is strict.
  • Candidates are concrete endpoint/model pairs in explicit order.
  • Named users cannot inherit single-user fallback consent.
  • Endpoint ownership and model allowlists are enforced before credential resolution.
  • Only documented availability failures may advance before substantive output.
  • Request, authentication, authorization, model, schema, and configuration failures do not advance.
  • Empty successful responses do not trigger fallback.
  • Accepted output or a completed tool call pins the route for later Agent rounds.
  • Streaming Chat, non-stream Chat, Agent events, history, metrics, and usage identify the route that actually answered.
  • Legacy default_model_fallbacks remains stored but is not used as foreground fallback policy.

Validation

Final rebase audit was performed against current dev.

  • Clean one-commit history on top of merged fix(model-routing): keep user-selected models strict #5801.
  • Focused foreground routing and Agent provenance tests passed.
  • Reconciliation-sensitive Agent/Chat regression tests passed.
  • Full authoritative Python suite passed in a clean virtual environment.
  • Python compilation passed.
  • JavaScript syntax checks passed.
  • No conflict markers remain.
  • Stale _ody_general_no_tool_mode reference removed.

How to Test

  1. Install requirements.txt in a clean virtual environment.
  2. Run the foreground routing, Agent provenance, stream-error, context-compaction,
    prefs, and LLM fallback tests.
  3. Run the full Python test suite with python -m pytest -q.
  4. Run python -m compileall -q core routes src.
  5. Run node --check against the modified/new Chat JavaScript modules.
  6. Confirm disabled policy stays strict and eligible availability failures alone
    may advance to an explicitly configured route.
  7. Confirm Agent fallback routes remain pinned after accepted output/tool effects
    and saved metrics identify the actual answering route.

Visual / UI changes — REQUIRED if you touched anything that renders

This changes existing Chat provenance and terminal-stream behavior rather than
introducing a new visual component or layout. Existing visual styling is
preserved. Browser-side JavaScript regression tests cover provenance, terminal
errors, background terminal state, and interrupted streams.

No new visual surface is introduced.

Make foreground fallback an explicit per-user, availability-only policy shared by streaming Chat, non-stream Chat, and Agent runs.

Preserve strict defaults, owner/model and credential boundaries, pinned Agent routes, and truthful per-round provenance/accounting. Carry provider-reported model identifiers through native streaming adapters, non-stream responses, and caches, and keep legacy default_model_fallbacks as tombstoned raw storage that generic settings APIs and agent tools cannot expose or mutate.
@github-actions github-actions Bot added the ready for review Description complete — ready for maintainer review label Aug 12, 2026
Comment thread static/js/chatRenderer.js
try {
const runCosts = JSON.parse(localStorage.getItem(_COST_RUNS_KEY) || '{}');
const sessionRuns = runCosts[sid] && typeof runCosts[sid] === 'object'
? runCosts[sid]
Comment thread static/js/chatRenderer.js
if (entries.length > _MAX_COST_RUNS_PER_SESSION) {
const overflow = entries.slice(0, entries.length - _MAX_COST_RUNS_PER_SESSION);
const costs = JSON.parse(localStorage.getItem(_COST_KEY) || '{}');
costs[sid] = (costs[sid] || 0) + overflow.reduce(
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready for review Description complete — ready for maintainer review

Projects

None yet

Development

Successfully merging this pull request may close these issues.

refactor(models): centralize explicit foreground fallback policy

3 participants