feat(supertonic,#434): Supertonic-3 TTS backend — converter, runtime, full wiring

Native ggml port of the ONNX-only Supertone/supertonic-3 release
(OpenRAIL-M weights, MIT sample code as the blueprint):

- models/convert-supertonic3-to-gguf.py: 4 ONNX graphs -> ONE GGUF with
  the unicode indexer, NFKD/ccc tables (from Python unicodedata), all 10
  preset voices, and every graph-baked constant verified by hard asserts
  (CFG 4/3, rotary theta, time freqs, attn scales, edge-pad amounts).
- src/supertonic_tts.{h,cpp}: text preprocessing (NFKD via GGUF tables),
  CPU duration predictor + text encoder (VITS relative attention, GST
  cross-attn with tanh keys, attention2 residual from the ORIGINAL text),
  ggml graphs for the vector estimator (CFG fused: v = 4c - 3u, Euler
  xt += v/steps, length-normalised rotary, EDGE-padded ConvNeXt) and the
  vocoder (edge pads, folded BatchNorm, PReLU-as-leaky, 512 samples/frame).
- tools/reference_backends/supertonic_tts.py: onnxruntime reference with
  promoted intermediates + seeded noise; supertonic_tts_diff() compares
  per stage with cos AND |mine|/|ref| norms; crispasr-diff branch wired.
- CLI adapter (--voice = preset F1..F5/M1..M5, --tts-speed, --tts-steps),
  factory + GGUF arch detect, session C-ABI (open/synthesize/voice/seed/
  steps/sample-rate/free/list), registry entry (license printed), and a
  quantizer allow-list (vf.*/voc.* 2-D matmuls only).

Unvalidated as of this commit: builds only syntax-checked locally (8 GB
box, no local builds); per-stage diff + TTS->ASR roundtrip run on Kaggle
next. docs/supertonic/PLAN.md carries the full graph-derived notes.

Author: crispasr integration

docs/supertonic/PLAN.md

@@ -0,0 +1,92 @@
+# Supertonic-3 TTS port (#434)
+
+## NOW — active work
+
+- [x] Licence verified from HF card 2026-09-12: model = OpenRAIL-M (`license: openrail`
+  tag on Supertone/supertonic-3, not gated), sample code MIT. Attribution +
+  responsible-use restrictions → registry `license` field must carry it.
+- [x] Blueprint read line-by-line: `supertone-inc/supertonic` `py/example_onnx.py` +
+  `py/helper.py` (MIT). Upstream pipeline reproduced locally with onnxruntime
+  (3.1 s clip, `M1` voice) as ground truth.
+- [x] All four ONNX graphs reverse-engineered (see notes below).
+- [ ] Converter → GGUF (single file, all voices embedded)
+- [ ] Reference dumper (onnxruntime intermediates, fixed noise)
+- [ ] C++ runtime + 12-point wiring
+- [ ] Kaggle build + diff + TTS→ASR roundtrip
+- [ ] HF upload + registry
+
+## Model facts (from onnx/tts.json + graph inspection, NOT guessed)
+
+- 4 ONNX components, opset 19, all feed-forward/deterministic:
+  `duration_predictor` (4 MB), `text_encoder` (36 MB), `vector_estimator`
+  (257 MB), `vocoder` (101 MB). 44.1 kHz output, 31 languages, 10 preset
+  voices (`voice_styles/{F1..F5,M1..M5}.json`: style_ttl [1,50,256],
+  style_dp [1,8,16]).
+- Text processing: NFKD → emoji strip → punctuation replacements → append "."
+  if no trailing punctuation → wrap `<lang>`…`</lang>` → per-CHARACTER unicode
+  codepoint → `unicode_indexer.json` (list[65536] → id, vocab 8322).
+- Chunking: 300 chars max (120 for ko/ja), 0.3 s silence joins.
+
+### duration_predictor
+char_emb(8322×64) *mask → prepend learned sentence_token (CLS at pos 0, mask
+extended with 1) → 6× ConvNeXt-1D (k5 dil 1..1, masked, LN eps per graph,
+GELU-erf, gamma) → VITS attn encoder ×2 layers (2 heads, head 32,
+emb_rel_k/v [1,9,32] window 4, post-LN, ReLU FFN k1 convs, masked) with
+GLOBAL residual: out = attn_out + convnext_out → slice CLS pos → 1×1 conv
+(64→64, no bias) → concat [sent(64) | style_dp.flat(128)] → Gemm 192→128 →
+PReLU → Gemm 128→1 → Exp = seconds. Then dur /= speed (default 1.05).
+
+### text_encoder
+char_emb(8322×256) → 6× ConvNeXt (dil 1,1,2,2,4,4) → VITS attn ×4 (4 heads,
+head 64, emb_rel [1,9,64]) → global residual add → *mask →
+speech_prompted_text_encoder: 2× GST cross-attn (2 heads, split axis2 →
+stack axis0): Q=text(256→256), K=tanh(W_k·style_key_prototype[1,50,256]),
+V=W_v·style_ttl, scores/scale, softmax over 50, out_fc, *mask, residual;
+then LayerNorm at end. Output text_emb [B,256,L].
+
+### vector_estimator (flow matching, CFG INSIDE the graph)
+- Batch doubled: cond half + uncond half (text→text_special_token bcast,
+  style k/v→style_{key,value}_special_token).
+- t = current_step/total_step; time emb: sin/cos(t·1000·freqs[32]),
+  freqs_i = 10000^(−i/31) → MLP 64→256 →Mish→ 64.
+- proj_in conv1x1 144→512 (no bias), *mask.
+- 4 blocks, flat module list per block:
+  [convnext×4 (dil 1,2,4,8) | time: x+=Linear64→512(t_emb) | convnext×1 |
+   attn: rotary cross-attn text (8 heads, head 64, Q=x·mask, K/V=text_emb,
+   angle=(arange(L)/actual_len)·theta_i, theta_i=10·10000^(−i/32),
+   rotate-half 32|32, scores/16, mask=−inf, out_fc, *mask, residual,
+   post-LN eps 1e-6) | convnext×1 | attention: GST style cross-attn
+   (2 heads, Q=x·mask 512→256, K=tanh(style_key), V=style_value, out 512,
+   residual, post-LN)]
+- last_convnext ×4 (dil 1,1,1,1) → proj_out conv1x1 512→144 (no bias), *mask.
+- Update (Euler, INSIDE graph): v = 4·v_cond − 3·v_uncond;
+  xt ← (xt + (1/total_step)·v)·latent_mask.
+- Noise: xt0 = randn(B,144,ceil(dur·sr/3072)) · latent_mask (chunk 512·6).
+
+### vocoder
+latent [B,144,L] → denorm (normalizer scale 0.25, ae.latent_mean/std
+[1,24,1]) + decompress 144→24 ch × 6L frames → conv k7 24→512 → 10×
+ConvNeXt (k7, dil 1,2,4,1,2,4,1,1,1,1, sym pad) → BatchNorm(512) →
+head: conv k3 512→2048 → PReLU → conv k1 2048→512 (no bias) → reshape:
+each frame = 512 samples → wav [B, 6L·512]. Trim to sr·dur.
+
+### Constants verified from the graphs
+- CFG: 4.0 / 3.0. Rotary theta[32] = 10·10000^(−i/32). Attn score /16.
+- Time freqs[32] = 10000^(−i/31), t scale 1000. Mish in time MLP.
+- ConvNeXt dwconv pads symmetric 2·dil (k5) / 3·dil (k7) — EDGE (replicate) mode in ALL four graphs, incl. vocoder embed + head layer1; zero-mode pads exist only inside VITS relative attention.
+- softmax mask value −inf; post-softmax re-mask with 0 (VITS style).
+
+## Design decisions
+- ONE GGUF: all 4 nets + unicode indexer (i32[65536]) + NFKD decomposition
+  tables (generated from Python unicodedata at convert time) + all 10 voices
+  (`voice.<name>.{ttl,dp}`). arch = "supertonic-tts".
+- CFG = two vector-field passes per step (cond/uncond share xt); 8 steps
+  default ⇒ 16 VF passes.
+- Diff harness: reference = onnxruntime with intermediates promoted to
+  graph outputs; fixed np.random noise saved in ref.gguf and injected into
+  the C++ side for parity (production uses its own RNG; CRISPASR_SEED-able).
+- Acceptance: TTS→ASR roundtrip (HARD RULE #3), on Kaggle CUDA + CPU.
+
+## Worktree
+`.claude/worktrees/feat-434-supertonic`, branch `feat/434-supertonic`.
+Kaggle account: chr1str (chr1s4 taken by a parallel agent).

examples/cli/CMakeLists.txt

@@ -53,6 +53,7 @@ add_executable(${TARGET}
     crispasr_backend_indextts.cpp
     crispasr_backend_f5_tts.cpp
     crispasr_backend_irodori_tts.cpp
+    crispasr_backend_supertonic.cpp
     crispasr_backend_pocket_tts.cpp
     crispasr_backend_csm.cpp
     crispasr_backend_parler.cpp
@@ -273,6 +274,7 @@ target_link_libraries(crispasr-diff PRIVATE
     parler-tts
     mini-omni2
     dots-tts
+    supertonic-tts
     confucius4-tts
     miotts
     miocodec
@@ -384,6 +386,7 @@ target_link_libraries(${TARGET} PRIVATE
     dots-tts
     confucius4-tts
     irodori-tts
+    supertonic-tts
     pocket-tts
     csm-tts
     voxcpm2_tts

examples/cli/crispasr_backend.cpp

@@ -80,6 +80,7 @@ std::unique_ptr<CrispasrBackend> crispasr_make_outetts_backend();
 std::unique_ptr<CrispasrBackend> crispasr_make_zonos_backend();
 std::unique_ptr<CrispasrBackend> crispasr_make_f5_tts_backend();
 std::unique_ptr<CrispasrBackend> crispasr_make_irodori_tts_backend();
+std::unique_ptr<CrispasrBackend> crispasr_make_supertonic_backend();
 std::unique_ptr<CrispasrBackend> crispasr_make_bark_backend();
 std::unique_ptr<CrispasrBackend> crispasr_make_pocket_tts_backend();
 std::unique_ptr<CrispasrBackend> crispasr_make_speecht5_backend();
@@ -213,6 +214,9 @@ std::unique_ptr<CrispasrBackend> crispasr_create_backend(const std::string& name
         return crispasr_make_f5_tts_backend();
     if (name == "irodori-tts" || name == "irodori_tts" || name == "irodori")
         return crispasr_make_irodori_tts_backend();
+    if (name == "supertonic" || name == "supertonic-tts" || name == "supertonic_tts" || name == "supertonic3" ||
+        name == "supertonic-3")
+        return crispasr_make_supertonic_backend();
     if (name == "pocket-tts" || name == "pocket_tts" || name == "pockettts" || name == "pocket" ||
         name == "pocket-tts-de" || name == "pocket-tts-german" || name == "pocket-tts-es" ||
         name == "pocket-tts-spanish" || name == "pocket-tts-it" || name == "pocket-tts-italian" ||

examples/cli/crispasr_backend_supertonic.cpp

@@ -0,0 +1,125 @@
+// crispasr_backend_supertonic.cpp — adapter for Supertonic-3 TTS (#434).
+//
+// Non-autoregressive flow-matching TTS (Supertone/supertonic-3, OpenRAIL-M),
+// 44.1 kHz, 31 languages, 10 preset voices embedded in the GGUF (F1..F5,
+// M1..M5). No voice cloning — the open release ships fixed style presets.
+//
+// Usage:
+//   crispasr --backend supertonic -m auto --tts "Hello world." -o out.wav
+//   crispasr --backend supertonic -m auto --tts "Guten Tag." -l de \
+//            --voice F2 --tts-speed 1.2 --tts-steps 8 -o out.wav
+
+#include "crispasr_backend.h"
+#include "whisper_params.h"
+
+#include "supertonic_tts.h"
+
+#include <cstdio>
+#include <cstdlib>
+#include <cstring>
+#include <string>
+#include <vector>
+
+namespace {
+
+class SupertonicBackend : public CrispasrBackend {
+public:
+    SupertonicBackend() = default;
+    ~SupertonicBackend() override { SupertonicBackend::shutdown(); }
+
+    const char* name() const override { return "supertonic"; }
+
+    uint32_t capabilities() const override { return CAP_TTS | CAP_AUTO_DOWNLOAD | CAP_TTS_SPEED; }
+
+    std::vector<crispasr_segment> transcribe(const float* /*samples*/, int /*n*/, int64_t /*t_off*/,
+                                             const whisper_params& /*params*/) override {
+        return {}; // TTS-only backend
+    }
+
+    bool init(const whisper_params& p) override {
+        supertonic_context_params sp = supertonic_context_default_params();
+        sp.n_threads = p.n_threads;
+        sp.verbosity = p.no_prints ? 0 : 1;
+        sp.use_gpu = p.use_gpu;
+        sp.seed = (uint64_t)(p.seed > 0 ? p.seed : 0);
+        if (p.tts_num_steps > 0)
+            sp.total_steps = p.tts_num_steps;
+        if (p.tts_speed > 0.0f && p.tts_speed != 1.0f) {
+            // upstream default is 1.05; treat --tts-speed as a multiplier on it
+            sp.speed = 1.05f * p.tts_speed;
+        }
+        ctx_ = supertonic_init_from_file(p.model.c_str(), sp);
+        if (!ctx_) {
+            std::fprintf(stderr, "crispasr[supertonic]: failed to load model '%s'\n", p.model.c_str());
+            return false;
+        }
+        apply_language(p);
+        apply_voice(p);
+        return true;
+    }
+
+    std::vector<float> synthesize(const std::string& text, const whisper_params& p) override {
+        if (!ctx_ || text.empty())
+            return {};
+        apply_language(p);
+        apply_voice(p);
+        if (p.tts_num_steps > 0)
+            supertonic_set_total_steps(ctx_, p.tts_num_steps);
+        if (p.tts_speed > 0.0f)
+            supertonic_set_speed(ctx_, 1.05f * p.tts_speed);
+        int n = 0;
+        float* pcm = supertonic_synthesize(ctx_, text.c_str(), &n);
+        if (!pcm || n <= 0)
+            return {};
+        std::vector<float> out(pcm, pcm + n);
+        std::free(pcm);
+        return out;
+    }
+
+    int tts_sample_rate() const override { return ctx_ ? supertonic_sample_rate(ctx_) : 44100; }
+
+    void shutdown() override {
+        if (ctx_) {
+            supertonic_free(ctx_);
+            ctx_ = nullptr;
+        }
+    }
+
+private:
+    void apply_language(const whisper_params& p) {
+        if (p.language.empty() || p.language == "auto")
+            return;
+        if (supertonic_set_language(ctx_, p.language.c_str()) != 0 && !p.no_prints && p.language != last_warn_lang_) {
+            std::fprintf(stderr,
+                         "crispasr[supertonic]: language '%s' not supported (en ko ja ar bg cs da de el es et fi fr "
+                         "hi hr hu id it lt lv nl pl pt ro ru sk sl sv tr uk vi); keeping current\n",
+                         p.language.c_str());
+            last_warn_lang_ = p.language;
+        }
+    }
+    // --voice selects a PRESET name (F1..F5, M1..M5); a path is rejected with
+    // a hint since this backend has no cloning.
+    void apply_voice(const whisper_params& p) {
+        if (p.tts_voice.empty() || p.tts_voice == last_voice_)
+            return;
+        if (supertonic_set_voice(ctx_, p.tts_voice.c_str()) != 0) {
+            if (!p.no_prints)
+                std::fprintf(stderr,
+                             "crispasr[supertonic]: unknown voice '%s' — expected a preset name F1..F5 / M1..M5 "
+                             "(Supertonic-3 has no voice cloning); keeping '%s'\n",
+                             p.tts_voice.c_str(), last_voice_.empty() ? "M1" : last_voice_.c_str());
+        } else {
+            last_voice_ = p.tts_voice;
+        }
+    }
+
+    supertonic_context* ctx_ = nullptr;
+    std::string last_voice_;
+    std::string last_warn_lang_;
+};
+
+} // namespace
+
+std::unique_ptr<CrispasrBackend> crispasr_make_supertonic_backend() {
+    return std::make_unique<SupertonicBackend>();
+}

examples/cli/crispasr_diff_main.cpp

@@ -95,6 +95,7 @@
 #include "tada_encoder.h"
 #include "tada_tts.h"
 #include "dots_tts.h"
+#include "supertonic_tts.h"
 #include "t5_translate.h"
 #include "miocodec.h"
 #include "miotts.h"
@@ -1254,6 +1255,11 @@ int main(int argc, char** argv) {
     // dots-tts: self-contained per-stage parity checks (no audio needed). The
     // reference is the isolated component dump from
     // tools/reference_backends/dots_tts_reference.py.
+    // supertonic-tts (#434): self-contained — ref carries text/voice/steps
+    // and the seeded noise; audio arg is ignored.
+    if (backend_name == "supertonic-tts" || backend_name == "supertonic") {
+        return supertonic_tts_diff(model_path.c_str(), ref_path.c_str(), /*verbosity=*/2);
+    }
     if (backend_name == "dots-tts") {
         int rp = dots_tts_penc_diff(model_path.c_str(), ref_path.c_str(), /*verbosity=*/2);
         int rd = dots_tts_dit_diff(model_path.c_str(), ref_path.c_str(), /*verbosity=*/2);

examples/crispasr-quantize/main.cpp

@@ -265,6 +265,14 @@ static bool crispasr_model_quantize(const std::string& fname_inp, const std::str
     // run on it but do not expect much.
     const bool is_btc = (arch == "btc");
 
+    // Supertonic-3 (#434): quantize ONLY the big vector-field / vocoder 2-D
+    // matmul weights (vf.* / voc.*). Everything else — the CPU-side duration
+    // predictor + text encoder (dp.* / te.*, tiny), the char embedders, the
+    // baked style prototypes / uncond tokens, all voice.* presets and the
+    // text.* index tables — stays at source precision. The dwconv kernels are
+    // 3-D and already fall out via ok_dims.
+    const bool is_supertonic = (arch == "supertonic-tts");
+
     const bool is_chatterbox =
         (arch.find("chatterbox") != std::string::npos || arch.find("kartoffelbox") != std::string::npos);
     // CosyVoice3: the three sub-models live in separate GGUFs but share the
@@ -798,6 +806,8 @@ static bool crispasr_model_quantize(const std::string& fname_inp, const std::str
             // `--q4_k` silently falls back to Q4_0. That fallback is why the q4
             // row costs so much -- it is Q4_0, not a k-quant.
             !(is_tabcnn && sname == "head.weight") &&
+            // Supertonic-3: allow-list vf.*/voc.* (see is_supertonic note).
+            !(is_supertonic && !(sname.rfind("vf.", 0) == 0 || sname.rfind("voc.", 0) == 0)) &&
             !(is_granite_family && !granite_quant_all && sname.find("enc.") == 0) &&
             // MOSS-Audio: keep encoder + adapter + deepstack at F16
             !(arch == "moss_audio" &&