Skip to content

Commit 4c79f43

Browse files
gary149claude
andauthored
Remove deprecated and underperforming models from catalog (#2398)
* chore(models): drop 15 models no longer on the router; refresh providers Remove model entries from prod.yaml/dev.yaml whose ids are no longer returned by router.huggingface.co/v1/models. These were dead overrides (models.ts only applies MODELS entries to ids present in the router response), so they never rendered. Removed: pearl-ai/Gemma-4-31B-it-pearl, zai-org/GLM-4.7-FP8, EssentialAI/rnj-1-instruct, moonshotai/Kimi-K2-Thinking, Qwen/QwQ-32B, deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B, katanemo/Arch-Router-1.5B, meta-llama/Llama-3.2-1B-Instruct, deepseek-ai/DeepSeek-R1-Distill-Qwen-32B, Qwen/Qwen3-Coder-480B-A35B-Instruct-FP8, meta-llama/Meta-Llama-3-70B-Instruct, NousResearch/Hermes-2-Pro-Llama-3-8B, baidu/ERNIE-4.5-300B-A47B-Base-PT, Sao10K/L3-70B-Euryale-v2.1, deepseek-ai/DeepSeek-Prover-V2-671B. Bump @huggingface/inference 4.11.3 -> 4.13.21 so PROVIDERS_HUB_ORGS includes deepinfra and nvidia; deepinfra now serves many router models and its avatar/label was missing. This covers all 13 providers the router currently uses. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DnzEAqrEE7qdFtWhSpWG3G * docs(skill): teach add-model-descriptions to prune deprecated models Extend the model-sync skill to compare the router list against the config in both directions: missing (add) and deprecated (remove). Adds a scripted two-way diff, a new removal step, and guidance to exclude the synthetic router alias (omni) and to flag—not silently drop—deprecated ids still referenced by TASK_MODEL / LLM_ROUTER_* env vars. Documents that MODELS is an overrides map (models.ts only applies entries present in /v1/models), so pruning dead overrides is a no-op at runtime. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DnzEAqrEE7qdFtWhSpWG3G * docs(skill): rename add-model-descriptions to sync-models The skill now syncs the model config in both directions (add new models, prune deprecated ones), so "add-model-descriptions" no longer describes it. Rename the directory and the frontmatter name/heading to sync-models, and update the self-referential /add-model-descriptions trigger to /sync-models. No other files reference the old name. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DnzEAqrEE7qdFtWhSpWG3G * fix(models): restore two entries the router serves again pearl-ai/Gemma-4-31B-it-pearl and zai-org/GLM-4.7-FP8 were pruned against an earlier router snapshot, but the router now returns both again (verified against router.huggingface.co/v1/models). Since MODELS entries are overrides merged onto router-provided models, dropping them wouldn't hide these models — it would only strip their metadata (notably GLM-4.7-FP8's supportsReasoning flag). Re-add both in their original positions. The other 13 removed ids remain absent from the router. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DnzEAqrEE7qdFtWhSpWG3G --------- Co-authored-by: Claude <noreply@anthropic.com>
1 parent f62079c commit 4c79f43

5 files changed

Lines changed: 131 additions & 76 deletions

File tree

.claude/skills/add-model-descriptions/SKILL.md renamed to .claude/skills/sync-models/SKILL.md

Lines changed: 54 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -1,11 +1,18 @@
11
---
2-
name: add-model-descriptions
3-
description: Add descriptions for new models from the HuggingFace router to chat-ui configuration, flag reasoning-capable ones, and enable artifacts for models with 32B+ parameters. Use when new models are released on the router and need descriptions added to prod.yaml and dev.yaml. Triggers on requests like "add new model descriptions", "update models from router", "sync models", or when explicitly invoking /add-model-descriptions.
2+
name: sync-models
3+
description: Sync chat-ui's model config with the HuggingFace router — add descriptions for new models, flag reasoning-capable ones, enable artifacts for models with 32B+ parameters, and prune deprecated models the router no longer serves. Use when models are released or removed on the router and prod.yaml/dev.yaml need syncing. Triggers on requests like "add new model descriptions", "update models from router", "sync models", "remove deprecated models", "prune models no longer on the router", or when explicitly invoking /sync-models.
44
---
55

6-
# Add Model Descriptions
6+
# Sync Models
77

8-
Add descriptions for new models available in the HuggingFace router to chat-ui's `prod.yaml` and `dev.yaml`. Also flag models that support the OpenAI-compatible `reasoning_effort` parameter so chat-ui shows the thinking-effort selector for them, and enable artifacts for models with 32B or more total parameters.
8+
Add descriptions for new models available in the HuggingFace router to chat-ui's `prod.yaml` and `dev.yaml`. Also flag models that support the OpenAI-compatible `reasoning_effort` parameter so chat-ui shows the thinking-effort selector for them, and enable artifacts for models with 32B or more total parameters. Finally, **prune deprecated models** — entries in the config whose ids the router no longer returns.
9+
10+
The comparison runs in two directions:
11+
12+
- **Missing** = in the router but not in the config → add them (steps 4–8).
13+
- **Deprecated** = in the config but no longer in the router → remove them (step 9).
14+
15+
Both directions share the same source of truth: whatever `https://router.huggingface.co/v1/models` returns _right now_.
916

1017
## Workflow
1118

@@ -19,13 +26,34 @@ Add descriptions for new models available in the HuggingFace router to chat-ui's
1926

2027
2. **Read current configuration**
2128

22-
- Read `chart/env/prod.yaml`
29+
- Read `chart/env/prod.yaml` (and `chart/env/dev.yaml` — the two share the same model set)
2330
- Extract model IDs from the `MODELS` JSON array in `envVars`
2431

25-
3. **Identify missing models**
26-
Compare router models with prod.yaml. Missing = in router but not in prod.yaml.
32+
3. **Identify missing and deprecated models**
33+
Compare the router list against the config in both directions:
34+
35+
- **Missing** = in router but not in the config → candidates to add (continue to step 4).
36+
- **Deprecated** = in the config but not in the router → candidates to remove (jump to step 9).
37+
38+
Compute both sets with a script so the diff is exact rather than eyeballed — the list is long and near-duplicate ids (`GLM-4.7` vs `GLM-4.7-FP8`, `-Instruct` vs `-Thinking`) are easy to miss:
39+
40+
```bash
41+
python3 - <<'EOF'
42+
import json, re, subprocess
43+
raw = subprocess.check_output(["curl","-sS","https://router.huggingface.co/v1/models"]).decode()
44+
router = {m["id"] for m in json.loads(raw)["data"]}
45+
txt = open("chart/env/prod.yaml").read()
46+
block = re.search(r"MODELS:\s*>\s*\n(.*?)\n\S", txt, re.S).group(1)
47+
ids = [e["id"] for e in json.loads(block)]
48+
keep = {"omni"} # never treat the router alias (PUBLIC_LLM_ROUTER_ALIAS_ID) as deprecated
49+
print("MISSING (add): ", sorted(router - set(ids)))
50+
print("DEPRECATED (rm): ", [i for i in ids if i not in router and i not in keep])
51+
EOF
52+
```
53+
54+
**Only operate on the missing set for the add/research steps (4–8).** Never edit, re-flag, or re-describe entries that already exist in `prod.yaml` / `dev.yaml` — even if you think their reasoning capability or description could be improved. Existing entries are intentionally curated and may have been hand-tuned for known quirks. Out of scope unless the user explicitly asks for a re-audit.
2755
28-
**Only operate on these missing models for the rest of the workflow.** Never edit, re-flag, or re-describe entries that already exist in `prod.yaml` / `dev.yaml` — even if you think their reasoning capability or description could be improved. Existing entries are intentionally curated and may have been hand-tuned for known quirks. Out of scope unless the user explicitly asks for a re-audit.
56+
**Exclude the router alias from the deprecated set.** The `omni` entry (whatever `PUBLIC_LLM_ROUTER_ALIAS_ID` is set to) is a synthetic alias, not a router model, so it never appears in `/v1/models`. Never remove it. The same goes for any other intentionally-synthetic id that isn't meant to come from the router.
2957
3058
4. **Research each missing model**
3159
For each missing model, search the web for its specifications:
@@ -90,7 +118,7 @@ Add descriptions for new models available in the HuggingFace router to chat-ui's
90118
- `"Vision-language Qwen for documents, GUI agents, and visual reasoning."`
91119
- `"Mobile agent for multilingual Android device automation."`
92120
93-
8. **Update both files**
121+
8. **Add new models to both files**
94122
Add new models at the TOP of the MODELS array in:
95123
96124
- `chart/env/prod.yaml`
@@ -115,15 +143,26 @@ Add descriptions for new models available in the HuggingFace router to chat-ui's
115143
116144
`supportsReasoning` is what makes chat-ui render the Thinking-effort dropdown in the chat footer for that model and forward `reasoning_effort` to the router. `supportsArtifacts` enables the artifacts side panel for the model.
117145
118-
9. **Commit changes**
119-
In the commit message, mention how many of the new models are reasoning-capable and how many get artifacts so it's easy to review.
120-
```
121-
git add chart/env/prod.yaml chart/env/dev.yaml
122-
git commit -m "feat: add descriptions for N new models from router (M reasoning-capable, K with artifacts)"
123-
```
146+
9. **Remove deprecated models from both files**
147+
Delete the full entry line for every id in the deprecated set (step 3) from **both** `chart/env/prod.yaml` and `chart/env/dev.yaml`. Match on the exact `"id"` value so near-duplicate ids aren't removed by accident, and keep the removal symmetric — the two files must end with the same model set.
148+
149+
Removing a deprecated entry is safe and low-risk: `MODELS` is an **overrides map**, not the model list. `src/lib/server/models.ts` builds the catalog from the router's `/v1/models` response and only applies a `MODELS` entry when its id is present in that response (it maps over the router models and looks each up in the override map). An entry whose id the router no longer serves is a **dead override** — it never renders in the UI — so pruning it changes nothing at runtime; it just keeps the config honest and readable.
150+
151+
After editing, re-parse the `MODELS` block in each file as JSON to confirm it's still valid and that no deprecated id remains (reuse the script from step 3 — the deprecated set should now be empty).
152+
153+
Do **not** touch models that are merely referenced by env vars but still present in the router (e.g. `TASK_MODEL`, `LLM_ROUTER_TOOLS_MODEL`, `LLM_ROUTER_MULTIMODAL_MODEL`). If a deprecated id _is_ referenced by one of those env vars, stop and flag it to the user instead of silently removing it — that indicates a config that needs a replacement model, not just a pruned line.
154+
155+
10. **Commit changes**
156+
In the commit message, mention how many models were added (and how many of those are reasoning-capable / get artifacts) and how many deprecated models were removed, so it's easy to review.
157+
```
158+
git add chart/env/prod.yaml chart/env/dev.yaml
159+
git commit -m "feat: sync models from router (+N added, M reasoning-capable, K artifacts, -D removed)"
160+
```
124161
125162
## Notes
126163
164+
- Prune-only runs are normal: the router may drop models without adding any. If the missing set is empty but the deprecated set isn't, skip steps 4–8 entirely and just do the removal (step 9) plus commit.
165+
- Add vs. remove are judged per-id: the router routinely keeps a base model while dropping its FP8/variant sibling (or the reverse). Treat `GLM-4.7` and `GLM-4.7-FP8`, or `-Instruct` and `-Thinking`, as independent ids — presence of one says nothing about the other. Trust the script's diff, not the family name.
127166
- FP8 variants: describe as "FP8 [base model] for efficient inference with [key capability]". If the base model is reasoning-capable, the FP8 variant is too — flag both. Same for artifacts: quantization doesn't change the parameter count, so a 32B+ base means the FP8 variant gets `supportsArtifacts` too.
128167
- Vision models: mention "vision-language" and key visual tasks. A vision model can still be reasoning-capable (e.g. `Qwen3-VL-*-Thinking`) — judge by the same rules.
129168
- Agent models: mention "agent" and automation capabilities.

chart/env/dev.yaml

Lines changed: 0 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -123,7 +123,6 @@ envVars:
123123
{ "id": "MiniMaxAI/MiniMax-M2.1", "description": "MoE agent model with multilingual coding and fast outputs."},
124124
{ "id": "zai-org/AutoGLM-Phone-9B-Multilingual", "description": "Mobile agent for multilingual Android device automation." },
125125
{ "id": "utter-project/EuroLLM-22B-Instruct-2512", "description": "European multilingual model for all EU languages and translation." },
126-
{ "id": "EssentialAI/rnj-1-instruct", "description": "8B code and STEM model rivaling larger models on agentic coding, math, and tool use." },
127126
{ "id": "MiniMaxAI/MiniMax-M2", "description": "Compact MoE model tuned for fast coding, agentic workflows, and long-context chat." , "supportsReasoning": true },
128127
{ "id": "Qwen/Qwen3-VL-30B-A3B-Instruct", "description": "Flagship Qwen3 vision-language model for high-accuracy image, text, and video reasoning." },
129128
{ "id": "Qwen/Qwen3-VL-30B-A3B-Thinking", "description": "Thinking-mode Qwen3-VL that emits detailed multimodal reasoning traces for difficult problems."},
@@ -132,7 +131,6 @@ envVars:
132131
{ "id": "allenai/Olmo-3-7B-Instruct", "description": "Lightweight Olmo assistant for instruction following, Q&A, and everyday open-source workflows." },
133132
{ "id": "deepcogito/cogito-671b-v2.1", "description": "Frontier-scale 671B MoE focused on deep reasoning, math proofs, and complex coding."},
134133
{ "id": "deepcogito/cogito-671b-v2.1-FP8", "description": "FP8 Cogito v2.1 making 671B-scale reasoning more affordable to serve and experiment with."},
135-
{ "id": "moonshotai/Kimi-K2-Thinking", "description": "Reasoning-focused Kimi K2 variant for deep chain-of-thought and large agentic tool flows." , "supportsReasoning": true },
136134
{ "id": "openai/gpt-oss-safeguard-20b", "description": "Safety-focused gpt-oss variant for content classification, policy enforcement, and LLM output filtering." , "supportsReasoning": true },
137135
{ "id": "zai-org/GLM-4.5", "description": "Flagship GLM agent model unifying advanced reasoning, coding, and tool-using capabilities."},
138136
{ "id": "zai-org/GLM-4.5V-FP8", "description": "FP8 vision-language GLM-4.5V for efficient multilingual visual QA, understanding, and hybrid reasoning."},
@@ -168,21 +166,15 @@ envVars:
168166
{ "id": "zai-org/GLM-4.5-Air", "description": "Efficient GLM strong reasoning and tool use at lower cost."},
169167
{ "id": "Qwen/Qwen2.5-7B-Instruct", "description": "Compact instruction model solid for basic conversation and tasks." },
170168
{ "id": "Qwen/Qwen3-32B", "description": "General purpose Qwen strong for complex queries and dialogues." },
171-
{ "id": "Qwen/QwQ-32B", "description": "Preview Qwen showcasing next generation features and alignment."},
172169
{ "id": "Qwen/Qwen3-235B-A22B-Instruct-2507", "description": "Flagship instruction Qwen near state of the art across domains." },
173170
{ "id": "meta-llama/Llama-3.3-70B-Instruct", "description": "Improved Llama alignment and structure powerful complex conversations." },
174-
{ "id": "deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B", "description": "Tiny distilled Qwen stepwise math and logic reasoning."},
175171
{ "id": "Qwen/Qwen3-235B-A22B", "description": "Qwen base at flagship scale ideal for custom fine tuning." },
176172
{ "id": "Qwen/Qwen2.5-Coder-32B-Instruct", "description": "Code model strong generation and tool use bridges sizes." },
177-
{ "id": "katanemo/Arch-Router-1.5B", "description": "Lightweight router model directs queries to specialized backends." },
178-
{ "id": "meta-llama/Llama-3.2-1B-Instruct", "description": "Ultra small Llama handles basic Q and A and instructions." },
179173
{ "id": "deepseek-ai/DeepSeek-R1-Distill-Qwen-7B", "description": "Distilled Qwen excels at stepwise logic in compact footprint."},
180174
{ "id": "deepseek-ai/DeepSeek-V3", "description": "General language model direct answers strong creative and knowledge tasks." },
181175
{ "id": "deepseek-ai/DeepSeek-V3-0324", "description": "Updated V3 better reasoning and coding strong tool use." },
182176
{ "id": "CohereLabs/command-a-translate-08-2025", "description": "Translation focused Command model high quality multilingual translation." },
183-
{ "id": "deepseek-ai/DeepSeek-R1-Distill-Qwen-32B", "description": "Distilled from R1 strong reasoning standout dense model."},
184177
{ "id": "baidu/ERNIE-4.5-VL-424B-A47B-Base-PT", "description": "Multimodal base text image pretraining for cross modal understanding." },
185-
{ "id": "Qwen/Qwen3-Coder-480B-A35B-Instruct-FP8", "description": "Quantized giant coder faster lighter retains advanced code generation." },
186178
{ "id": "deepseek-ai/DeepSeek-R1-0528", "description": "R1 update improved reasoning, fewer hallucinations, adds function calling.", "parameters": { "max_tokens": 32000 }},
187179
{ "id": "Qwen/Qwen3-14B", "description": "Balanced Qwen good performance and efficiency for assistants." },
188180
{ "id": "MiniMaxAI/MiniMax-M1-80k", "description": "Long context MoE very fast excels at long range reasoning and code."},
@@ -193,23 +185,18 @@ envVars:
193185
{ "id": "Qwen/Qwen2.5-72B-Instruct", "description": "Accurate detailed instruction model supports tools and long contexts." },
194186
{ "id": "meta-llama/Llama-Guard-4-12B", "description": "Safety guardrail model filters and enforces content policies." },
195187
{ "id": "CohereLabs/command-a-vision-07-2025", "description": "Command model with image input captioning and visual QA." },
196-
{ "id": "meta-llama/Meta-Llama-3-70B-Instruct", "description": "Instruction tuned Llama improved reasoning and reliability over predecessors." },
197-
{ "id": "NousResearch/Hermes-2-Pro-Llama-3-8B", "description": "Small Hermes highly steerable maximized helpfulness for basics." },
198188
{ "id": "Sao10K/L3-8B-Stheno-v3.2", "description": "Community Llama variant themed tuning and unique conversational style." },
199189
{ "id": "CohereLabs/c4ai-command-r-08-2024", "description": "Cohere Command variant instruction following with specialized tuning." },
200-
{ "id": "baidu/ERNIE-4.5-300B-A47B-Base-PT", "description": "Large base model foundation for specialized language systems." },
201190
{ "id": "CohereLabs/aya-expanse-32b", "description": "Aya Expanse large comprehensive knowledge and reasoning capabilities." },
202191
{ "id": "CohereLabs/c4ai-command-a-03-2025", "description": "Updated Command assistant improved accuracy and general usefulness." },
203192
{ "id": "CohereLabs/command-a-reasoning-08-2025", "description": "Command variant optimized for complex multi step logical reasoning." },
204193
{ "id": "alpindale/WizardLM-2-8x22B", "description": "Multi expert WizardLM MoE approach for efficient high quality generation." },
205194
{ "id": "deepseek-ai/DeepSeek-R1-Distill-Llama-70B", "description": "Llama distilled from R1 improved reasoning enterprise friendly."},
206-
{ "id": "Sao10K/L3-70B-Euryale-v2.1", "description": "Creative community instruct model with distinctive persona." },
207195
{ "id": "CohereLabs/aya-vision-32b", "description": "Larger Aya Vision advanced vision language with detailed reasoning." },
208196
{ "id": "CohereLabs/c4ai-command-r7b-arabic-02-2025", "description": "Command tuned for Arabic fluent and culturally appropriate outputs." },
209197
{ "id": "Sao10K/L3-8B-Lunaris-v1", "description": "Community Llama creative role play oriented themed persona." },
210198
{ "id": "meta-llama/Llama-3.1-70B-Instruct", "description": "Instruction tuned Llama improved reasoning and factual reliability." },
211199
{ "id": "zai-org/GLM-4-32B-0414", "description": "Open licensed GLM matches larger proprietary models on benchmarks." },
212-
{ "id": "deepseek-ai/DeepSeek-Prover-V2-671B", "description": "Specialist for mathematical proofs and formal reasoning workflows."},
213200
{ "id": "Qwen/Qwen2.5-Coder-3B-Instruct", "description": "Tiny coding assistant basic code completions and explanations." }
214201
]
215202

0 commit comments

Comments
 (0)