Add 16GB-tier MoE models to gen-3 eval set + regenerate dashboard (#107)

* Add 16GB-tier MoE models to v0.7.5 eval set (LFM2.5, Mellum2)

Wire LFM2.5-8B-A1B and Mellum2-12B-A2.5B (Thinking + Instruct) into the
batch eval harness and sampling registry, and fold their n=50 results
into the v0.7.5 / gen 3 dataset. No version bump — gen is a comparability
epoch, not a release version, and these are net-new configs.

- sampling_defaults: official card params for the three models
- batch_eval: GGUF entries, "new-models" config subset, reasoning-format
  auto flags for the two CoT models (LFM2.5, Mellum2 Thinking)
- eval_results_v0.7.5.jsonl: +15,600 rows stamped gen=3
  (Mellum2-12B Instruct native/reforged leads at 81.0%)
- EVAL_GUIDE: document eval generations + the post-release addendum

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* Regenerate dashboard with 16GB-tier additions + fix regen docs

Rebuild the shipped dashboard and markdown views from the full versioned
dataset set (v0.6.0 + v0.7.0 + v0.7.4 + v0.7.5) so the new gen-3 16GB-tier
models (LFM2.5-8B-A1B, Mellum2-12B Thinking + Instruct) appear alongside the
carried-forward older generations. Mellum2-12B Instruct (native, reforged)
leads the additions at 81.0%.

Also correct EVAL_GUIDE: the shipped dashboard is a multi-file render across
all eval_results_v*.jsonl (dedup_latest_gen keeps newest gen per config), not
a single-file render. The prior single-file example silently dropped every
model absent from that one file, including the carried-forward generations.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Author: antoinezambelli

docs/EVAL_GUIDE.md

@@ -161,13 +161,26 @@ Resume is automatic: re-run the same command and it skips completed scenarios.
 
 ### Committed datasets
 
-Released datasets are versioned in the repo: `eval_results_vX.Y.Z.jsonl` (LFS-tracked). The current shipped dashboard at `docs/results/dashboard.html` reflects the latest version. To regenerate the dashboard or markdown views against a specific release:
+Released datasets are versioned in the repo: `eval_results_vX.Y.Z.jsonl` (LFS-tracked). The current shipped dashboard at `docs/results/dashboard.html` reflects the latest version. The shipped dashboard is built from **all** versioned datasets at once — `report` merges them and `dedup_latest_gen` keeps the newest gen per config, so older-gen models carry forward as superscript-badged peers. Pass every `eval_results_v*.jsonl`, oldest to newest:
 
 ```bash
-python -m tests.eval.report eval_results_v0.7.0.jsonl --html docs/results/dashboard.html --markdown docs/results/
+python -m tests.eval.report \
+  eval_results_v0.6.0.jsonl eval_results_v0.7.0.jsonl \
+  eval_results_v0.7.4.jsonl eval_results_v0.7.5.jsonl \
+  --html docs/results/dashboard.html --markdown docs/results/
 ```
 
-Older datasets (e.g. `eval_results_v0.6.0.jsonl`) remain in the repo for comparison and reproducibility. `batch_eval` writes to `eval_results.jsonl` by default; rename to a versioned filename before committing to the repo.
+Generating from a single file is fine for a quick look at one release in isolation, but it drops every model not present in that file — including the carried-forward older generations — so do not commit a single-file render as the shipped dashboard. `batch_eval` writes to `eval_results.jsonl` by default; rename to a versioned filename before committing to the repo.
+
+### Eval generations and post-release addenda
+
+The `gen` field (an integer injected per-row, legend in `report.py:GEN_INFO`) is a **comparability epoch, not a release version**. It is bumped only when a change is judged eval-material; many releases can share one gen, and a single gen can span several eval waves merged across files (`dedup_latest_gen` keeps the newest gen per config). This decouples "did we add models / re-sweep" from "did we cut a release" — adding models does not require a version bump.
+
+To fold new models into an existing dataset, stamp them with that dataset's `gen` and append the rows. Because they are net-new configs, no existing number is recomputed and they slot into the leaderboard as same-gen peers (no carry-forward badge).
+
+Addenda to date:
+
+- **2026-06-17 — v0.7.5 / gen 3, 16GB tier (rig-01):** added `LFM2.5-8B-A1B-Q4_K_M` and `Mellum2-12B-A2.5B-Q4_K_M` (Thinking + Instruct), each native + prompt, reforged + bare, n=50, into `eval_results_v0.7.5.jsonl`. Run post-release on llama.cpp `b9647`. No version bump — stamped `gen: 3`. Headline: Mellum2-12B Instruct (native, reforged) 81.0%. Caveat: replay coverage for these three is `none` only, not the full none/keep-last/full grid the other gen-3 8–14B models carry.
 
 ### Forge eval report
 

docs/results/dashboard.html

@@ -17,4 +17,4 @@ For model and backend recommendations, see [Model Guide](../MODEL_GUIDE.md).
 - [native-vs-prompt.md](raw/native-vs-prompt.md) — llama-server native FC vs prompt-injected, reforged only
 - [reasoning-replay.md](raw/reasoning-replay.md) — reasoning_replay policy comparison (none / keep-last / full) per config
 
-*Generated 2026-06-11 20:28*
+*Generated 2026-06-18 02:22*

docs/results/index.md

@@ -1,5 +1,38 @@
 # Forge Eval — Native vs Prompt (llama-server)
 
+## LFM2.5-8B-A1B-Q4_K_M
+
+```
+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+Model/Backend                             Scr      Acc      Cmp    Eff   Wst    Spd     N    rel   arg   tsl   b2s   s3s   crt   srn   err   dgr   dge   art   grs   iar rel_s arg_s tsl_s b2s_s s3s_s crt_s srn_s err_s dgr_s dge_s art_s grs_s iar_s
+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+LFM2.5-8B-A1B-Q4_K_M LS/N [reforged]    57.7%    64.9%    88.8%    68%   1.3   6.6s    50    100    98    90    60   100    82   100   100    18     4     0     6     0   100    98    88    78   100    50   100   100    18     2     0     8     0
+LFM2.5-8B-A1B-Q4_K_M LS/P [reforged]    35.3%    48.7%    72.5%    48%   2.7  19.1s    50    100    54     0    24    66     8    92    62    44     2     0     4     4   100    40     0    48    60     6    84    70    44     2     0     4     0
+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+```
+
+## Mellum2-12B-A2.5B-Instruct-Q4_K_M
+
+```
+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+Model/Backend                                          Scr      Acc      Cmp    Eff   Wst    Spd     N    rel   arg   tsl   b2s   s3s   crt   srn   err   dgr   dge   art   grs   iar rel_s arg_s tsl_s b2s_s s3s_s crt_s srn_s err_s dgr_s dge_s art_s grs_s iar_s
+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+Mellum2-12B-A2.5B-Instruct-Q4_K_M LS/N [reforged]    81.0%    82.3%    98.4%    76%   1.5   1.2s    50    100   100   100    86   100   100   100   100   100    44     0    98    24   100   100   100    88   100   100   100   100   100    44     0    98    24
+Mellum2-12B-A2.5B-Instruct-Q4_K_M LS/P [reforged]    60.7%    62.4%    97.3%    95%   0.2   1.2s    50     94   100   100   100   100    42    98   100    16    24     0     6    10    92   100   100   100   100    34    98   100    26    22     0    10     6
+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+```
+
+## Mellum2-12B-A2.5B-Thinking-Q4_K_M
+
+```
+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+Model/Backend                                          Scr      Acc      Cmp    Eff   Wst    Spd     N    rel   arg   tsl   b2s   s3s   crt   srn   err   dgr   dge   art   grs   iar rel_s arg_s tsl_s b2s_s s3s_s crt_s srn_s err_s dgr_s dge_s art_s grs_s iar_s
+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+Mellum2-12B-A2.5B-Thinking-Q4_K_M LS/N [reforged]    68.4%    68.7%    99.5%    80%   0.7   8.8s    50    100    92   100   100   100    44   100    96    56     0     2    38    86   100    82   100   100   100    34   100    86    64     0     0    30    68
+Mellum2-12B-A2.5B-Thinking-Q4_K_M LS/P [reforged]    66.8%    68.4%    97.8%    94%   0.3   7.9s    50    100   100   100   100   100    42    92   100    28    24     4    32    38   100   100   100   100   100    52    90   100    40    30     2    46    18
+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+```
+
 ## Ministral-3-14B-Instruct-2512-Q4_K_M
 
 ```
@@ -252,4 +285,4 @@ Eval generations (older runs carried forward, superscript-tagged):
   ¹ gen 1 — v0.6.0 suite — incl. Anthropic ablation (commit 2b05dc4, 2026-05-08)
   ² gen 2 — v0.7.0 lineup refresh (8–14B) + 32GB tier debut (v0.7.4) (commit 655e1f6, 2026-05-22)
 
-*Generated 2026-06-11 20:28*
+*Generated 2026-06-18 02:22*

docs/results/raw/native-vs-prompt.md

@@ -417,4 +417,4 @@ Eval generations (older runs carried forward, superscript-tagged):
   ¹ gen 1 — v0.6.0 suite — incl. Anthropic ablation (commit 2b05dc4, 2026-05-08)
   ² gen 2 — v0.7.0 lineup refresh (8–14B) + 32GB tier debut (v0.7.4) (commit 655e1f6, 2026-05-22)
 
-*Generated 2026-06-11 20:28*
+*Generated 2026-06-18 02:22*