Skip to content

Commit 772f042

Browse files
docs: add CHANGELOG for v0.4.3 prefill release (#569)
Highlight Qwythos 128k (+183.6% vs llama) and Qwen3.6 32k (+82.7%) prefill wins, plus Qwen3.6 on sparkinfer.com/chat.
1 parent 9303036 commit 772f042

1 file changed

Lines changed: 83 additions & 0 deletions

File tree

CHANGELOG.md

Lines changed: 83 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,89 @@
33
Notable changes to sparkinfer. Format loosely follows [Keep a Changelog](https://keepachangelog.com);
44
versions track the GitHub [releases](https://github.com/gittensor-ai-lab/sparkinfer/releases).
55

6+
## [0.4.3] — 2026-07-21
7+
8+
sparkinfer's **prefill stack is the headline of this release**: Qwythos (Qwen3.5) is now
9+
**+183.6% faster than llama.cpp at 128k prefill**, and Qwen3.6 is **+82.7% faster at 32k prefill**
10+
both on the same RTX 5090 / pinned llama.cpp commit (`6f4f53f`). The live chat at
11+
**[sparkinfer.com/chat](https://sparkinfer.com/chat)** now serves **Qwen3.6**.
12+
13+
### ⚡ Prefill — the main story
14+
15+
Dense + MoE prefill landed a chain of weight-amortized and int8 tensor-core wins since v0.4.2.
16+
Long-context prompt processing is where agents feel latency first — this release closes that gap
17+
past llama.cpp at the lengths that matter.
18+
19+
#### Qwythos (Qwen3.5-9B) · Q4_K_M · RTX 5090
20+
21+
| context | sparkinfer (pp tok/s) | llama.cpp (pp tok/s) | vs llama |
22+
|---|---:|---:|---:|
23+
| **4k prefill** | **~19,580** | 11,105 | **~+76%** |
24+
| **32k prefill** | **~20,170** | 9,772 | **~+106%** |
25+
| **64k prefill** | **~20,150** | 8,154 | **~+147%** |
26+
| **128k prefill** | **~17,015** | 6,000 | **+183.6%** |
27+
28+
Headline: **128k prefill +183.6% vs llama.cpp** (17,015 vs 5,999.59 pp tok/s; #557).
29+
30+
#### Qwen3.6-35B-A3B · UD-Q4_K_M · RTX 5090
31+
32+
Expert-grouped int8 MoE prefill (#530#537#548#553) lifts long-prompt MoE far past the
33+
v0.4.2 decode-focused frontier.
34+
35+
| context | sparkinfer (pp tok/s) | llama.cpp (pp tok/s) | vs llama |
36+
|---|---:|---:|---:|
37+
| **512 prefill** | **~3,510** | 8,737 | (short-N still climbing) |
38+
| **4k prefill** | **~11,580** | 8,726 | **~+33%** |
39+
| **16k prefill** | **~14,160** | 8,390 | **~+69%** |
40+
| **32k prefill** | **~14,587** | 7,984 | **+82.7%** |
41+
42+
Headline: **32k prefill +82.7% vs llama.cpp** (14,587 vs 7,984 pp tok/s).
43+
44+
### 🌐 Chat — Qwen3.6 on sparkinfer.com
45+
46+
| | |
47+
|---|---|
48+
| **Chat** | [sparkinfer.com/chat](https://sparkinfer.com/chat) — now includes **Qwen3.6** |
49+
| **Website** | [sparkinfer.com](https://sparkinfer.com/) |
50+
| **Demo API** | [api.sparkinfer.com](https://api.sparkinfer.com/) — OpenAI-compatible |
51+
52+
### Prefill optimizations landed since v0.4.2
53+
54+
- **#531** (`eval:XL`) — faithful batched Qwythos prefill through 128k
55+
- **#530** (`eval:XL`) — batched weight-amortized MoE prefill for Qwen3.6
56+
- **#537** (`eval:XL`) — expert-grouped int8 MoE prefill (large long-ctx jump)
57+
- **#552** (`eval:L`) — `mma.sync` bf16 prefill GEMM for dense long-ctx
58+
- **#548** (`eval:XL`) — chunk-parallel GDN scan + faster prefill dequant/GEMM
59+
- **#557** (`eval:XL`) — selective int8 FFN+attn at long ctx — **128k +183.6% vs llama**
60+
- **#553** (`eval:XL`) — single-pass Q→i8 row dequant for Qwen3.6 short-N
61+
- **#561** (`eval:XS`) — expert-group L2 MoE prefill for short-N (N≤512)
62+
63+
### Eval harness & trust
64+
65+
- **#529** — bidir prefill scoring for Qwen3.5 and Qwen3.6
66+
- **#564** — REJECT when any no-regression gate fails
67+
- **#567** — tighter H2 long-context bars (top1≥0.90, KL≤0.5)
68+
- **#568** — copycat guard: skip main-shared tiny helpers
69+
70+
### What changed since v0.4.2
71+
72+
| headline | v0.4.2 | v0.4.3 | shift |
73+
|---|---:|---:|---|
74+
| Qwythos prefill at 128k | ~6,888 pp/s (~+15% vs llama) | **~17,015 pp/s (+183.6% vs llama)** | **~2.5×** |
75+
| Qwen3.6 prefill at 32k | ~1,282 pp/s (behind llama) | **~14,587 pp/s (+82.7% vs llama)** | **~11×** |
76+
| Live chat | Qwythos-focused demo | **Qwen3.6 on [sparkinfer.com/chat](https://sparkinfer.com/chat)** | new |
77+
78+
**Verified:** RTX 5090 · Qwythos prefill **~17,015 pp/s at 128k (+183.6% vs llama)** · Qwen3.6 prefill
79+
**~14,587 pp/s at 32k (+82.7% vs llama)** · Polaris-attested eval logs · llama.cpp `6f4f53f`.
80+
81+
### Contributors
82+
83+
- **@Paral1995**#531 (batched Qwythos to 128k), #552 (bf16 `mma.sync` GEMM), #557 (int8 long-ctx FFN+attn)
84+
- **@James-CUDA**#537 (expert-grouped int8 MoE), #561 (short-N L2 MoE groups)
85+
- **@inference2026**#530 (weight-amortized MoE prefill), #553 (Q→i8 row dequant)
86+
- **@fansilas**#548 (chunk-parallel GDN + prefill dequant/GEMM)
87+
- **@skyrocket2026**#529 (bidir prefill scoring), #564/#567/#568 (eval gates + copycat), dashboard + release
88+
689
## [0.4.2] — 2026-07-17
790

891
sparkinfer now **beats llama.cpp on Qwythos prefill at every tracked context** — climbing from **290 → 16,083 pp tok/s**

0 commit comments

Comments
 (0)