Remove deprecated and underperforming models from catalog#2398
Conversation
Remove model entries from prod.yaml/dev.yaml whose ids are no longer returned by router.huggingface.co/v1/models. These were dead overrides (models.ts only applies MODELS entries to ids present in the router response), so they never rendered. Removed: pearl-ai/Gemma-4-31B-it-pearl, zai-org/GLM-4.7-FP8, EssentialAI/rnj-1-instruct, moonshotai/Kimi-K2-Thinking, Qwen/QwQ-32B, deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B, katanemo/Arch-Router-1.5B, meta-llama/Llama-3.2-1B-Instruct, deepseek-ai/DeepSeek-R1-Distill-Qwen-32B, Qwen/Qwen3-Coder-480B-A35B-Instruct-FP8, meta-llama/Meta-Llama-3-70B-Instruct, NousResearch/Hermes-2-Pro-Llama-3-8B, baidu/ERNIE-4.5-300B-A47B-Base-PT, Sao10K/L3-70B-Euryale-v2.1, deepseek-ai/DeepSeek-Prover-V2-671B. Bump @huggingface/inference 4.11.3 -> 4.13.21 so PROVIDERS_HUB_ORGS includes deepinfra and nvidia; deepinfra now serves many router models and its avatar/label was missing. This covers all 13 providers the router currently uses. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DnzEAqrEE7qdFtWhSpWG3G
Extend the model-sync skill to compare the router list against the config in both directions: missing (add) and deprecated (remove). Adds a scripted two-way diff, a new removal step, and guidance to exclude the synthetic router alias (omni) and to flag—not silently drop—deprecated ids still referenced by TASK_MODEL / LLM_ROUTER_* env vars. Documents that MODELS is an overrides map (models.ts only applies entries present in /v1/models), so pruning dead overrides is a no-op at runtime. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DnzEAqrEE7qdFtWhSpWG3G
The skill now syncs the model config in both directions (add new models, prune deprecated ones), so "add-model-descriptions" no longer describes it. Rename the directory and the frontmatter name/heading to sync-models, and update the self-referential /add-model-descriptions trigger to /sync-models. No other files reference the old name. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DnzEAqrEE7qdFtWhSpWG3G
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: f6bdcb84e6
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| { "id": "meta-llama/Llama-4-Scout-17B-16E-Instruct", "description": "Native multimodal 17B MoE with 16 experts and very long context."}, | ||
| { "id": "CohereLabs/c4ai-command-r7b-12-2024", "description": "Compact 7B Command R for multilingual RAG, tool use, and agents." }, | ||
| { "id": "pearl-ai/Gemma-4-31B-it-pearl", "description": "Community Gemma-4-31B variant integrated with Pearl mining for chain-validated inference."}, | ||
| { "id": "inclusionAI/Ling-2.6-1T", "description": "1T MoE with 50B active params, hybrid MLA-Linear attention, and fast thinking.", "supportsArtifacts": true, "parameters": { "max_tokens": 32768 } }, |
There was a problem hiding this comment.
Because src/lib/server/models.ts builds the catalog from ${OPENAI_BASE_URL}/models and only merges MODELS entries as overrides (see modelsRaw creation and the override merge around lines 249-301), deleting this entry from the production MODELS list does not hide the model; /api/v2/models only filters models where unlisted is true. For any removed ID still returned by the router (for example, HF's supported-models page still lists pearl-ai/Gemma-4-31B-it-pearl: https://huggingface.co/inference/models), production will keep showing and allowing it, just without the custom metadata. Keep an override with unlisted: true or add an explicit denylist if the intent is to remove it from the catalog.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Good catch on the mechanism — MODELS entries are overrides merged onto the router's /v1/models, so removing one can't hide a model the router still serves; it only drops the metadata. The intent here was never to hide router-served models, only to prune dead overrides for ids the router no longer returns, so unlisted: true/a denylist isn't the right tool.
That said, checking the actual endpoint chat-ui reads (https://router.huggingface.co/v1/models, the OPENAI_BASE_URL), two of the pruned ids — pearl-ai/Gemma-4-31B-it-pearl and zai-org/GLM-4.7-FP8 — are in fact being served again (my original snapshot was stale). For those, dropping the override would have stripped their metadata (notably GLM-4.7-FP8's supportsReasoning), so I've restored both in ed919e5. The other 13 pruned ids are genuinely absent from that endpoint, so removing their dead overrides is a no-op at runtime.
Note the supported-models page you linked is a broader catalog than the router endpoint chat-ui actually consumes, so it isn't a reliable signal for what this config should carry.
Generated by Claude Code
pearl-ai/Gemma-4-31B-it-pearl and zai-org/GLM-4.7-FP8 were pruned against an earlier router snapshot, but the router now returns both again (verified against router.huggingface.co/v1/models). Since MODELS entries are overrides merged onto router-provided models, dropping them wouldn't hide these models — it would only strip their metadata (notably GLM-4.7-FP8's supportsReasoning flag). Re-add both in their original positions. The other 13 removed ids remain absent from the router. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DnzEAqrEE7qdFtWhSpWG3G
2 participants
Summary
This PR removes 16 models from both development and production environment configurations that are either deprecated, underperforming, or no longer actively maintained.
Changes
Removed models from dev.yaml and prod.yaml:
pearl-ai/Gemma-4-31B-it-pearl- Community variant with Pearl miningzai-org/GLM-4.7-FP8- FP8 quantized GLM variantEssentialAI/rnj-1-instruct- 8B code/STEM modelmoonshotai/Kimi-K2-Thinking- Reasoning-focused Kimi variantQwen/QwQ-32B- Preview Qwen modeldeepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B- Tiny distilled modelkatanemo/Arch-Router-1.5B- Lightweight router modelmeta-llama/Llama-3.2-1B-Instruct- Ultra small Llamadeepseek-ai/DeepSeek-R1-Distill-Qwen-32B- Distilled dense modelQwen/Qwen3-Coder-480B-A35B-Instruct-FP8- Quantized giant codermeta-llama/Meta-Llama-3-70B-Instruct- Older Llama instruction modelNousResearch/Hermes-2-Pro-Llama-3-8B- Small Hermes variantbaidu/ERNIE-4.5-300B-A47B-Base-PT- Large base modelSao10K/L3-70B-Euryale-v2.1- Creative community modeldeepseek-ai/DeepSeek-Prover-V2-671B- Mathematical proofs specialistUpdated dependency:
@huggingface/inferencefrom^4.11.3to^4.13.21Notes
The model removals are applied consistently across both dev and prod environments to maintain parity between deployment configurations.
https://claude.ai/code/session_01DnzEAqrEE7qdFtWhSpWG3G