Skip to content

Commit 4970139

Browse files
docs: update for v2.3.0 release — changelog, sponsors, roadmap, zh-CN/zh-TW docs
1 parent 9d5a541 commit 4970139

6 files changed

Lines changed: 64 additions & 27 deletions

File tree

CHANGELOG.md

Lines changed: 29 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -5,6 +5,29 @@ All notable changes to Shimmy will be documented in this file.
55
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
66
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
77

8+
## [2.3.0] - 2026-07-20
9+
10+
### Added
11+
- **Grammar control hooks** — schoolmarm+grammar module for structured generation
12+
- **PPT invariant cage (B1-B3)** — golden-vault-based regression detection. Contracts verify per-layer RMS/checksum against vault oracles for 12 populated models. Run with `cargo test --test test_invariants -- --test-threads=1`.
13+
- **Discrete GPU preference** — adapter selection now enumerates all GPUs and prefers `DiscreteGpu` over `IntegratedGpu` (commit `6ed0349`). Fixes multi-GPU laptops picking the Intel iGPU.
14+
15+
### Fixed
16+
- **batch_count: 0 → batch_count: 1** in `server_inference.rs`. The QKV shader guard `if (global_id.y >= params.batch_count)` killed ALL threads when `batch_count=0` (since `0>=0` is true), producing no Q/K/V output. Root cause of the server-side gibberish regression (bisected to commit `43027d3`).
17+
- **Rust/WGSL struct layout mismatch** — per-field quant types (Q/QK/K/V) aligned between Rust `LayerParams` and WGSL shader structs.
18+
19+
### Changed
20+
- Airframe dependency updated to 0.2.9.
21+
- Shimmy version bumped to 2.3.0.
22+
23+
### Library API
24+
- `tokenizer_arc()` — shared tokenizer access
25+
- `eos_token()`, `im_end_token()` — token accessors
26+
- `fse_control_from_patterns()` — grammar pattern compiler
27+
- `trace_callback()` — capture hook for invariant probes
28+
29+
---
30+
831
## [2.2.1] - 2026-06-30
932

1033
### Fixed
@@ -1027,9 +1050,12 @@ The systematic engineering discipline established in v1.7.3 creates a **BULLETPR
10271050
### Free Forever Commitment
10281051
Shimmy is committed to being free forever with no asterisks, no "free for now" periods, and no pivot to paid services. The MIT license ensures this commitment is legally binding.
10291052

1030-
[Unreleased]: https://github.com/Michael-A-Kuykendall/shimmy/compare/v2.0.0...HEAD
1031-
[2.0.0]: https://github.com/Michael-A-Kuykendall/shimmy/compare/v1.10.0...v2.0.0
1032-
[1.10.0]: https://github.com/Michael-A-Kuykendall/shimmy/compare/v1.9.0...v1.10.0
1053+
[Unreleased]: https://github.com/Michael-A-Kuykendall/shimmy/compare/v2.3.0...HEAD
1054+
[2.3.0]: https://github.com/Michael-A-Kuykendall/shimmy/compare/v2.2.1...v2.3.0
1055+
[2.2.1]: https://github.com/Michael-A-Kuykendall/shimmy/compare/v2.2.0...v2.2.1
1056+
[2.2.0]: https://github.com/Michael-A-Kuykendall/shimmy/compare/v2.1.0...v2.2.0
1057+
[2.1.0]: https://github.com/Michael-A-Kuykendall/shimmy/compare/v2.0.0...v2.1.0
1058+
[2.0.0]: https://github.com/Michael-A-Kuykendall/shimmy/compare/v1.9.0...v2.0.0
10331059
[1.9.0]: https://github.com/Michael-A-Kuykendall/shimmy/compare/v1.8.1...v1.9.0
10341060
[1.8.1]: https://github.com/Michael-A-Kuykendall/shimmy/compare/v1.8.0...v1.8.1
10351061
[1.8.0]: https://github.com/Michael-A-Kuykendall/shimmy/compare/v1.7.4...v1.8.0

README.md

Lines changed: 5 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -62,9 +62,11 @@ Shimmy is a **single-binary** that provides **100% OpenAI-compatible endpoints**
6262

6363
**⚡ NEW in v2.1.0**: [TurboShimmy INT4 KV](#-turboshimmy-int4-kv)~7× less KV cache VRAM with one flag. Run Llama-3.2-3B on 4 GB GPUs.
6464

65-
## 🔥 Airframe Engine (v0.2.7)
65+
**⚡ NEW in v2.3.0**: Adapter selection fix (prefers discrete GPU over integrated), grammar control hooks, 357 passing tests.
6666

67-
Starting in v2.0.0, Shimmy's default inference engine is **Airframe** — a pure-Rust WebGPU (WGSL) transformer runtime built from scratch. **v0.2.7** brings the Inference Saturation Fabric (ISF) refit and TDR transport integration to production.
67+
## 🔥 Airframe Engine (v0.2.9)
68+
69+
Starting in v2.0.0, Shimmy's default inference engine is **Airframe** — a pure-Rust WebGPU (WGSL) transformer runtime built from scratch. **v0.2.9** brings the production batch_count fix, GPU adapter selection improvement, grammar hooks, and the PPT invariant cage.
6870

6971
**See [airframe CHANGELOG](https://github.com/Michael-A-Kuykendall/airframe/blob/master/CHANGELOG.md) for full release notes.**
7072

@@ -328,7 +330,7 @@ cd shimmy
328330
cargo build --release
329331
```
330332

331-
> **Note**: The Airframe GPU engine is a private dependency and **cannot be built from source** by public users. The [pre-built release binaries](#-download-pre-built-binaries-recommended) already include Airframe compiled in — download those to get full GPU acceleration. `cargo install shimmy` installs the huggingface engine variant from crates.io.
333+
> **Note**: The Airframe GPU engine is a public crate on [crates.io](https://crates.io/crates/airframe) and builds from source automatically. `cargo install shimmy` installs the huggingface engine variant from crates.io; for the full GPU build use the [pre-built release binaries](#-download-pre-built-binaries-recommended) or clone and build with `cargo build --release`.
332334
333335
### GPU Acceleration
334336

ROADMAP.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -34,8 +34,8 @@ Shimmy is a zero-config, OpenAI-compatible inference server with a native WebGPU
3434
- No CUDA toolkit or Vulkan SDK required; wgpu handles adapter selection
3535

3636
## 🎯 Q2–Q3 2026 Milestones
37-
- [ ] **Stop tokens from GGUF metadata** — read `tokenizer.ggml.eos_token_id` natively
38-
- [ ] **Quantization in Airframe** — Q4_K_M and Q8_0 inference on the WebGPU pipeline
37+
- [x] **Stop tokens from GGUF metadata** — read `tokenizer.ggml.eos_token_id` natively
38+
- [x] **Quantization in Airframe** — Q4_K_M and Q8_0 inference on the WebGPU pipeline
3939
- [ ] **SafeTensors support** — ingest `.safetensors` model checkpoints directly
4040
- [ ] **Multi-model serving** — load balancing across multiple active models
4141
- [ ] **Enterprise Embeddings**`/v1/embeddings` endpoint targeting RAG workloads

SPONSORS.md

Lines changed: 4 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,15 +1,14 @@
11
# These amazing people make Shimmy possible 🙏
22

3-
## Infrastructure Partners ($500+/month)
4-
*Your logo could be here*
5-
63
## Corporate Backers ($100+/month)
7-
*Your logo could be here*
4+
- **[ZephyrCloudIO](https://github.com/ZephyrCloudIO)**
85

96
## Bug Prioritizers ($25+/month)
7+
*Your name could be here — [sponsor](https://github.com/sponsors/Michael-A-Kuykendall)*
108

119
## Coffee Tier Heroes ($5+/month)
12-
*Your support could be here*
10+
- **[gqf2008](https://github.com/gqf2008)** (高庆丰)
11+
- **[alistairheath](https://github.com/alistairheath)**
1312

1413
---
1514

docs/USER_MANUAL.zh-CN.md

Lines changed: 12 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,7 @@
66

77
[📚 中文文档中心](zh-CN/README.md) · **简体中文手册** · [繁體中文](USER_MANUAL.zh-TW.md) · [English](../README.md)
88

9-
版本:v2.0.0 及以上 · 最后更新:2026 年 5
9+
版本:v2.3.0 及以上 · 最后更新:2026 年 7
1010

1111
</div>
1212

@@ -51,12 +51,17 @@ Shimmy 是一个**单一二进制文件**的本地 AI 推理服务器,提供
5151
| 🦀 **纯 Rust 实现** | 单一可执行文件,无 Python 运行时,无 C++ 依赖 |
5252
| 🖥️ **跨平台 GPU 加速** | 通过 WebGPU(wgpu)支持 NVIDIA、AMD、Intel 及 Apple Silicon |
5353

54-
### v2.0.0 新特性
54+
### v2.3.0 新特性
5555

56-
- **Airframe 引擎**:全新纯 Rust WGSL 推理引擎,彻底取代 llama.cpp
57-
- **WebGPU 加速**:通过 wgpu 自动选择最佳 GPU 适配器
58-
- **YaRN RoPE 缩放**:支持超长上下文窗口(最高 16384+ tokens)
59-
- **零工具链依赖**:无需 CUDA 工具包、Vulkan SDK 或任何 C++ 编译器
56+
- **独立 GPU 自动优先选择**:多 GPU 系统(如笔记本)自动优先选择独立显卡而非集成显卡
57+
- **Grammar 控制钩子**:支持结构化生成(schoolmarm+grammar 集成)
58+
- **PPT 不变性测试笼**:基于黄金 vault 的回归检测,覆盖 12 个模型的逐层 RMS/校验和验证
59+
- **batch_count 修复**:修复了 QKV 着色器中 batch_count=0 导致所有线程被终止的严重问题
60+
- **357 个库测试全部通过**
61+
62+
### 历史版本
63+
64+
**v2.0.0**: Airframe 引擎首发,纯 Rust WGSL 推理引擎,彻底取代 llama.cpp
6065

6166
---
6267

@@ -177,7 +182,7 @@ cargo install shimmy
177182
### 方式三:从源码构建(含 Airframe 引擎)
178183

179184
```bash
180-
git clone https://github.com/Michael-A-Kuykendall/shimmy --recurse-submodules
185+
git clone https://github.com/Michael-A-Kuykendall/shimmy
181186
cd shimmy
182187
cargo build --release --features airframe,huggingface
183188

docs/USER_MANUAL.zh-TW.md

Lines changed: 12 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,7 @@
66

77
[📚 中文文件中心](zh-TW/README.md) · [简体中文](USER_MANUAL.zh-CN.md) · **繁體中文** · [English](../README.md)
88

9-
版本:v2.0.0 及以上 · 最後更新:2026 年 5
9+
版本:v2.3.0 及以上 · 最後更新:2026 年 7
1010

1111
</div>
1212

@@ -51,12 +51,17 @@ Shimmy 是一個**單一執行檔**的本地 AI 推論伺服器,提供與 Open
5151
| 🦀 **純 Rust 實作** | 單一可執行檔,無 Python 執行環境,無 C++ 相依套件 |
5252
| 🖥️ **跨平台 GPU 加速** | 透過 WebGPU(wgpu)支援 NVIDIA、AMD、Intel 及 Apple Silicon |
5353

54-
### v2.0.0 新功能
54+
### v2.3.0 新功能
5555

56-
- **Airframe 引擎**:全新純 Rust WGSL 推論引擎,完全取代 llama.cpp
57-
- **WebGPU 加速**:透過 wgpu 自動選擇最佳 GPU 適配器
58-
- **YaRN RoPE 縮放**:支援超長上下文視窗(最高 16384+ tokens)
59-
- **零工具鏈相依**:無需 CUDA 工具包、Vulkan SDK 或任何 C++ 編譯器
56+
- **獨立 GPU 自動優先選擇**:多 GPU 系統(如筆電)自動優先選擇獨立顯示卡而非內建顯示卡
57+
- **Grammar 控制鉤子**:支援結構化生成(schoolmarm+grammar 整合)
58+
- **PPT 不變性測試籠**:基於黃金 vault 的迴歸檢測,覆蓋 12 個模型的逐層 RMS/校驗和驗證
59+
- **batch_count 修復**:修復了 QKV 著色器中 batch_count=0 導致所有執行緒被終止的嚴重問題
60+
- **357 個庫測試全部通過**
61+
62+
### 歷史版本
63+
64+
**v2.0.0**: Airframe 引擎首發,純 Rust WGSL 推論引擎,完全取代 llama.cpp
6065

6166
---
6267

@@ -177,7 +182,7 @@ cargo install shimmy
177182
### 方式三:從原始碼建置(含 Airframe 引擎)
178183

179184
```bash
180-
git clone https://github.com/Michael-A-Kuykendall/shimmy --recurse-submodules
185+
git clone https://github.com/Michael-A-Kuykendall/shimmy
181186
cd shimmy
182187
cargo build --release --features airframe,huggingface
183188

0 commit comments

Comments
 (0)