You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: CHANGELOG.md
+23-52Lines changed: 23 additions & 52 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -7,50 +7,15 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
7
7
8
8
## [Unreleased]
9
9
10
-
## [2.0.0] - 2026-05-20
10
+
## [2.0.0] - 2026-05-26
11
11
12
-
### 🚀 **v2.0 PUBLIC RELEASE** — Airframe GPU Engine + Submodule Distribution
12
+
### 🚀 **v2.0 PUBLIC RELEASE** — Airframe GPU Engine
13
13
14
-
This is the public release of Shimmy v2.0. The Airframe engine (introduced in v1.10.0 as an internal
15
-
milestone) is now the officially declared default. Distribution architecture has been updated: Airframe
16
-
ships as a git submodule in CI, enabling `--features airframe` release binaries for all platforms.
17
-
The crates.io package (`cargo install shimmy`) provides the huggingface engine variant; GitHub Releases
18
-
provide the full Airframe GPU binaries.
14
+
This release replaces the llama.cpp inference backend with **Airframe**, a pure-Rust WGSL GPU inference engine. Airframe runs the entire transformer pipeline on GPU using WebGPU compute shaders — no C++ dependencies, no build flags, no platform matrix headaches. This is a major architectural break from v1.x.
19
15
20
-
### What's New Since v1.10.0
16
+
The llama.cpp code path is **historically parked**, not deleted. It remains accessible via `--legacy` flag or `SHIMMY_ENGINE_BACKEND=llama` if needed.
21
17
22
-
**API**
23
-
- Added `POST /v1/completions` text completion endpoint (OpenAI-compatible)
24
-
-`POST /v1/chat/completions` now accepts `frequency_penalty` and `presence_penalty` fields; values map to `repeat_penalty = 1.0 + max(freq, presence) * 0.5`
25
-
- Input validation: empty `messages` array returns 400; `max_tokens` of 0 or above 131072 returns 400
26
-
27
-
**Inference**
28
-
- Fixed stop token propagation: `<|eot_id|>` (Llama-3) and other model-specific stop tokens from chat templates now correctly terminate generation; previously they were silently dropped and the model would generate until `max_tokens`
29
-
30
-
**Model Discovery**
31
-
- Added `~/.cache/lm-studio/models` to auto-discovery search paths (fixes #184)
32
-
33
-
**Distribution**
34
-
- Airframe added as a git submodule — CI now builds all release binaries with GPU engine included
35
-
-`publish = true`: `cargo install shimmy` now works (installs huggingface engine variant from crates.io)
36
-
- GitHub Releases provide platform binaries with the full Airframe WebGPU engine
37
-
38
-
**Versioning**
39
-
- Formal v2.0.0 semantic version bump to signal the architectural break from llama.cpp v1.x
40
-
- CHANGELOG moved from internal milestones to public release cadence
41
-
42
-
**Migration Guide**
43
-
- See `docs/MIGRATION_v2.md` for step-by-step migration from v1.x (llama.cpp) to v2.0 (Airframe)
This release replaces the llama.cpp inference backend with **Airframe**, a pure-Rust WGSL GPU inference engine. Airframe runs the entire transformer pipeline on GPU using WebGPU compute shaders with F32 precision throughout — no C++ dependencies, no build flags, no platform matrix headaches.
50
-
51
-
The llama.cpp code path is **historically parked**, not deleted. It remains accessible via `--legacy` flag or `SHIMMY_ENGINE_BACKEND=llama` for anyone who needs it, but it is no longer the default and will not receive new features.
52
-
53
-
### 🏆 What Changed
18
+
### What's New Since v1.9.0
54
19
55
20
**New Default Engine: Airframe**
56
21
- Pure Rust — no C++ toolchain required at runtime
@@ -60,32 +25,38 @@ The llama.cpp code path is **historically parked**, not deleted. It remains acce
60
25
- Stateless per-request inference with full KV cache reset between requests
61
26
- Model spec auto-derived from GGUF metadata — works with any GGUF-format model
62
27
28
+
**API**
29
+
- Added `POST /v1/completions` text completion endpoint (OpenAI-compatible)
30
+
-`POST /v1/chat/completions` now accepts `frequency_penalty` and `presence_penalty` fields; values map to `repeat_penalty = 1.0 + max(freq, presence) * 0.5`
31
+
- Input validation: empty `messages` array returns 400; `max_tokens` of 0 or above 131072 returns 400
32
+
33
+
**Inference**
34
+
- Fixed stop token propagation: `<|eot_id|>` (Llama-3) and other model-specific stop tokens from chat templates now correctly terminate generation; previously they were silently dropped and the model would generate until `max_tokens`
35
+
- Chunked prefill: prompts now processed in 512-token chunks (was one giant dispatch) — extended context viable again
36
+
63
37
**Model Configuration**
64
38
- Set `SHIMMY_BASE_GGUF=/path/to/model.gguf` to configure the default model
65
39
- Pass `--model-path /path/to/model.gguf` to the `serve` command as an alternative
66
40
- No more hardcoded developer paths shipped in the binary
67
41
68
-
**Chunked Prefill**
69
-
- Prefill now processes prompts in 512-token chunks (was one giant dispatch)
70
-
- Restores manageable GPU dispatch times at all context sizes
71
-
- Extended context (`SHIMMY_MAX_CTX=4096`, `8192`, etc.) is viable again
42
+
**Model Discovery**
43
+
- Added `~/.cache/lm-studio/models` to auto-discovery search paths (fixes #184)
72
44
73
45
**Template Routing Fixed**
74
46
- TinyLlama, Llama-1, and Llama-2 models now correctly use ChatML template
75
47
- Llama-3 template reserved for models explicitly named `llama-3` / `llama3`
76
48
77
-
### ⚠️ Migration Notes
49
+
**Distribution**
50
+
-`cargo install shimmy` now works — crates.io package uses huggingface engine variant
51
+
- GitHub Releases provide platform binaries with the full Airframe WebGPU engine
78
52
79
-
**llama.cpp is historically parked.** If you were using the default inference path:
80
-
- Everything you could do with TinyLlama via llama.cpp works identically through Airframe
81
-
- Opt back to llama.cpp with `--legacy` or `SHIMMY_ENGINE_BACKEND=llama` if needed
53
+
### ⚠️ Migration from v1.x
82
54
83
55
**SHIMMY_BASE_GGUF is now required.** The default model is no longer hardcoded. Set the env var or pass `--model-path`.
84
56
85
-
### 🔧 Internal Changes
86
-
- Removed `main_integration` stub module
87
-
- Cleaned all unused import warnings across the codebase
88
-
-`GpuRuntime::load()` now derives ModelSpec from GGUF metadata instead of hardcoded TinyLlama constants
57
+
**llama.cpp is historically parked.** Everything you could do with TinyLlama via llama.cpp works identically through Airframe. Opt back with `--legacy` or `SHIMMY_ENGINE_BACKEND=llama` if needed.
58
+
59
+
See `docs/MIGRATION_v2.md` for full migration steps.
0 commit comments