Skip to content

fix(models,engine): persist solo per-load overrides across restart; kv fp8 off for multimodal#51

Merged
webdevtodayjason merged 1 commit into
mainfrom
codex/fix-050-followups
Jul 6, 2026
Merged

fix(models,engine): persist solo per-load overrides across restart; kv fp8 off for multimodal#51
webdevtodayjason merged 1 commit into
mainfrom
codex/fix-050-followups

Conversation

@webdevtodayjason

Copy link
Copy Markdown
Contributor

Summary

Fixes the two fp8/override defects found live on the 0.5.0 fleet, plus a small AutoData report polish.

  • Solo-load override persistence: the primary model boots from config.json, which never received per-load overrides (feat(launch): served-model-name aliases + per-load config overrides #41 persisted them only to the stacked-instance manifest) — a VLM loaded with kv_cache_dtype=auto came back after restart on fp8 and produced corrupted output. Every solo load now persists the full override set to NodeConfig, with absent fields reset to defaults (no cross-load inheritance).
  • Adversarial review caught + fixed a would-be regression: the first cut leaked the previous primary's overrides (quantization, trust_remote_code, served_model_name…) onto the next model's launch config via the shared mutable config — reproduced against the branch, fixed via _resolved_overrides() applied to the actual launch config, with regression tests inspecting the real backend object. Also fixed the live-vs-persisted divergence on same-model reloads.
  • Multimodal fp8-KV auto-skip: fp8 KV corrupts VLM generation on GB10 (verified: garbage on fp8, perfect on auto; text models unaffected). The NVIDIA backend now defaults multimodal models (vision_config / VL architectures in the local config.json) to kv_cache_dtype=auto. Explicit operator fp8 is still honored via a new kv_cache_dtype_explicit provenance flag. Not-yet-downloaded repo-ids keep the fp8 default (documented ceiling).
  • AutoData valset meta reports now surface best_lift/best_score/best_significant.

639 passed, 1 xfailed; ruff clean. Review-gated (1 blocker + 2 majors found, all fixed with regression tests).

🤖 Generated with Claude Code

https://claude.ai/code/session_017NziXzqT1L9kj2T1byA3Ak

…efault kv fp8 off for multimodal

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017NziXzqT1L9kj2T1byA3Ak
@webdevtodayjason webdevtodayjason merged commit 66a8eee into main Jul 6, 2026
1 check passed
@webdevtodayjason webdevtodayjason deleted the codex/fix-050-followups branch July 6, 2026 18:18
@webdevtodayjason webdevtodayjason mentioned this pull request Jul 6, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant