Skip to content

Commit ba3499e

Browse files
committed
fix(release): ggml_backend_is_cuda is unlinkable in the shipped .so; gate cosyvoice3's tail per backend
Two things, both from the v0.8.33 tag run. 1. THE RELEASE BUILD FAILED TO LINK. build-linux-x86_64-cuda and -cuda13 both died with: libcrispasr.so.0.8.33: undefined reference to `ggml_backend_is_cuda' Same class as the ggml_backend_is_cpu break fixed earlier this cycle: ggml_backend_is_cuda is a CUDA-MODULE symbol and is not linkable into the shared library. Two call sites had it — mimo_tokenizer.cpp and parakeet.cpp. Both now test the backend NAME via ggml_backend_name(), which is core ggml, matching what granite_speech, dots_tts and omnivoice already do — copying the code that demonstrably links rather than inventing a fourth approach. ROCm is included for the same reason granite_speech includes it. Swept src/*.cpp for the rest of the class: the only other hit, ggml_backend_is_metal, is a comment. Every other tag workflow passed — Docker, language wrappers, Win CUDA13, Piper Windows, ISA fallback and all four bindings — so this is the release artifacts only, not the tag. 2. CRISPASR_COSYVOICE3_CAMPP_TAIL=legacy, a per-backend switch for the unresolved tail-divisor question. Both paths synthesise correctly (8/8 each), so this is a CONSISTENCY control, not correctness: by default `--voice ref.wav` differs from the baked voice bank by cos ~0.998 for the same voice, because the shipped bank was produced with the old divisor. `legacy` makes the two agree. Backend-specific ON PURPOSE. CRISPASR_CAMPP_LEGACY_SEGPOOL already exists but is global, and would drag chatterbox, confucius4, dots-tts and fireredtts3 away from their own settled PyTorch references to answer a question that is only open for cosyvoice3.
1 parent cdb6ab0 commit ba3499e

4 files changed

Lines changed: 64 additions & 4 deletions

File tree

docs/environment-variables.md

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1603,3 +1603,21 @@ end-to-end cosine cannot do.
16031603
- `CRISPASR_ZONOS_TTS_BENCH`
16041604
- `CRISPASR_ZONOS_TTS_TEXT`
16051605
- `CRISPASR_ZONOS_VULKAN_NATIVE`
1606+
1607+
### `CRISPASR_COSYVOICE3_CAMPP_TAIL`
1608+
1609+
`legacy` makes cosyvoice3's CAM++ speaker encoder use the old partial-tail
1610+
divisor. Default is the same convention as every other CAM++ backend.
1611+
1612+
Both paths synthesise correctly — the TTS→ASR roundtrip is 8/8 on each — so this
1613+
is a CONSISTENCY switch, not a correctness one. The eight speaker embeddings
1614+
baked into the shipped `cosyvoice3-voices.gguf` were produced with the old
1615+
divisor, so by default a voice cloned from a WAV and the same voice taken from
1616+
the bank differ by cos ~0.998. Set this to `legacy` if you need those two paths
1617+
to agree.
1618+
1619+
Why cosyvoice3 specifically is unsettled: its upstream is `campplus.onnx`, and
1620+
two onnxruntime builds disagree about `AveragePool(ceil_mode=1)` on the same
1621+
clip. `CRISPASR_CAMPP_LEGACY_SEGPOOL` also exists but is GLOBAL — it would drag
1622+
chatterbox, confucius4, dots-tts and fireredtts3 away from their own settled
1623+
PyTorch references to answer a cosyvoice3-only question.

src/cosyvoice3_tts.cpp

Lines changed: 31 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -3121,6 +3121,33 @@ ggml_tensor* cv3_dit_block_apply(ggml_context* ctx0, ggml_tensor* x, ggml_tensor
31213121
// runtime detector counts 133 per graph — q/k/v/o and both FFN layers across all
31223122
// 22 blocks — far more than reading the source suggested. Gated; the fold is an
31233123
// exact restatement (a linear is per-token independent). See core/quant_bcast.h.
3124+
// #431-adj: which CAM++ tail divisor THIS backend uses, selectable.
3125+
//
3126+
// cosyvoice3's upstream is campplus.onnx and the question is genuinely open:
3127+
// two onnxruntime builds disagree on AveragePool(ceil_mode=1) for the same
3128+
// clip, while the eight embeddings baked into the shipped cosyvoice3-voices.gguf
3129+
// match the LEGACY divisor exactly (cos 1.000000, |x| 14.1197 on zero_shot).
3130+
//
3131+
// Both paths synthesise correctly — the end-to-end roundtrip is 8/8 on each —
3132+
// so this is not a correctness switch, it is a CONSISTENCY one:
3133+
//
3134+
// default (fixed) matches the other four CAM++ backends and the torch
3135+
// reference; `--voice ref.wav` differs from the baked bank
3136+
// by cos ~0.998 for the same voice.
3137+
// legacy matches the baked voice bank, so a voice cloned from a WAV
3138+
// and the same voice taken from the bank agree.
3139+
//
3140+
// CRISPASR_COSYVOICE3_CAMPP_TAIL=legacy selects the second. Backend-specific on
3141+
// purpose: CRISPASR_CAMPP_LEGACY_SEGPOOL exists too, but it is global and would
3142+
// drag chatterbox, confucius4, dots-tts and fireredtts3 away from their own
3143+
// (settled) PyTorch references to fix a cosyvoice3-only question.
3144+
static campplus_segpool::tail_divisor cosyvoice3_campp_tail() {
3145+
const char* e = crispasr_env::get("CRISPASR_COSYVOICE3_CAMPP_TAIL");
3146+
if (e && (std::strcmp(e, "legacy") == 0 || std::strcmp(e, "kernel") == 0))
3147+
return campplus_segpool::tail_divisor::kernel_size;
3148+
return campplus_segpool::tail_divisor::window_width;
3149+
}
3150+
31243151
static inline ggml_tensor* CV3MM(ggml_context* c, ggml_tensor* w, ggml_tensor* x) {
31253152
// Default ON: verified on the shipped q4_k LLM + q8_0 flow — the detector
31263153
// reports 133 broadcasting quantized matmuls per graph without the fold and
@@ -5620,7 +5647,8 @@ bool cv3_extract_native_runtime_voice(cosyvoice3_tts_context* ctx, const char* w
56205647
return false;
56215648

56225649
std::vector<float> native_spk =
5623-
chatterbox_campplus::embed_speaker(ctx->campplus.model, ctx->campplus.cache, pcm16.data(), (int)pcm16.size());
5650+
chatterbox_campplus::embed_speaker(ctx->campplus.model, ctx->campplus.cache, pcm16.data(), (int)pcm16.size(),
5651+
/*stats_var_floor=*/0.0f, cosyvoice3_campp_tail());
56245652
if (native_spk.size() != 192)
56255653
return false;
56265654

@@ -6353,7 +6381,8 @@ extern "C" int cosyvoice3_tts_extract_spk_emb(struct cosyvoice3_tts_context* ctx
63536381
if (sr != 16000)
63546382
pcm = core_audio::resample_polyphase(pcm.data(), (int)pcm.size(), sr, 16000);
63556383
auto emb =
6356-
chatterbox_campplus::embed_speaker(ctx->campplus.model, ctx->campplus.cache, pcm.data(), (int)pcm.size());
6384+
chatterbox_campplus::embed_speaker(ctx->campplus.model, ctx->campplus.cache, pcm.data(), (int)pcm.size(),
6385+
/*stats_var_floor=*/0.0f, cosyvoice3_campp_tail());
63576386
if (emb.size() != 192)
63586387
return -1;
63596388
std::memcpy(out_spk_emb, emb.data(), 192 * sizeof(float));

src/mimo_tokenizer.cpp

Lines changed: 8 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -443,7 +443,14 @@ extern "C" struct mimo_tokenizer_context* mimo_tokenizer_init_from_file(const ch
443443
if (const char* e = std::getenv("CRISPASR_MIMO_TOK_CPU"); e && *e && *e != '0')
444444
weights_be = ctx->backend_cpu;
445445
#if defined(GGML_USE_CUDA)
446-
ctx->cuda_rvq_available = weights_be == ctx->backend && ggml_backend_is_cuda(ctx->backend);
446+
// ggml_backend_is_cuda() is a CUDA-MODULE symbol: linking it into
447+
// libcrispasr.so fails with "undefined reference" in the release CUDA
448+
// build. Test the backend NAME instead, which is core ggml and is what
449+
// granite_speech, dots_tts and omnivoice already do. ROCm is included
450+
// for the same reason granite_speech includes it.
451+
const char* be_name = ctx->backend ? ggml_backend_name(ctx->backend) : nullptr;
452+
ctx->cuda_rvq_available =
453+
weights_be == ctx->backend && be_name && (std::strstr(be_name, "CUDA") || std::strstr(be_name, "ROCm"));
447454
#endif
448455
core_gguf::WeightLoad wl;
449456
if (!core_gguf::load_weights(path_model, weights_be, "mimo_tokenizer", wl)) {

src/parakeet.cpp

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1404,7 +1404,13 @@ static bool parakeet_gpu_encoder_projection(const parakeet_context* ctx) {
14041404
if (const char* e = crispasr_env::get("CRISPASR_RNNT_GPU_ENC_PROJ"))
14051405
return *e == '1';
14061406
#if defined(GGML_USE_CUDA)
1407-
return ggml_backend_is_cuda(ctx->backend);
1407+
// ggml_backend_is_cuda() is a CUDA-MODULE symbol: linking it into
1408+
// libcrispasr.so fails with "undefined reference" in the release CUDA
1409+
// build. Test the backend NAME instead, which is core ggml and is what
1410+
// granite_speech, dots_tts and omnivoice already do. ROCm is included
1411+
// for the same reason granite_speech includes it.
1412+
const char* be_name = ctx->backend ? ggml_backend_name(ctx->backend) : nullptr;
1413+
return be_name && (std::strstr(be_name, "CUDA") || std::strstr(be_name, "ROCm"));
14081414
#else
14091415
(void)ctx;
14101416
return false;

0 commit comments

Comments
 (0)