Canary 1B Deep Dive
🌐 Language: English | Français
NVIDIA Canary-1B v2 is an attention encoder-decoder (AED) model that transcribes AND translates in a single forward pass. It's the recommended pick when your machine has at least 6 GB of VRAM and you want higher accuracy (lower WER than Parakeet) plus built-in translation without routing through Ollama, Google, or LibreTranslate.
The Rust port in dictee (src/canary.rs, src/model_canary.rs, src/decoder_canary.rs) was originally adapted from onnx-asr by Ivan Stupakov and is now fully self-contained — ONNX graphs are loaded directly via ort, tokenization uses the tokenizers crate (official HuggingFace), and decoder prompt construction is home-grown.
- Architecture
- Languages supported (7 in the Rust port)
- Native translation — the differentiator
- Audio pipeline
- VRAM & duration limits
- Native punctuation & capitalization
- GPU required in practice
- Source language must match audio
- File layout
- Benchmarks
Canary uses a classic encoder-decoder layout (similar to Whisper, but NVIDIA-optimized):
Audio (16 kHz mono)
↓
Mel-spectrogram (128 bins, n_fft=512, 10 ms hop, Hann)
↓
┌──────────────────────────────────┐
│ FastConformer encoder │
│ - Conv subsampling │
│ - Conformer blocks (multi-head) │
│ - Output: acoustic embeddings │
└──────────────────────────────────┘
↓
┌──────────────────────────────────┐
│ Transformer decoder │
│ - Cross-attention on encoder │
│ - Autoregressive generation │
│ - Prompt tokens: SOT + source │
│ + target + PNC + NOITN + … │
└──────────────────────────────────┘
↓
SentencePiece vocabulary
↓
Text with native punctuation & capitalization
Key difference vs Parakeet: Canary is an encoder-decoder, not a transducer. It generates output tokens autoregressively by cross-attending over encoder embeddings, whereas Parakeet-TDT emits (token, duration) pairs in a single pass. Practical consequences:
- Canary is more accurate (lower WER on every shared language)
- Canary is slower (autoregressive vs parallel TDT)
- Canary supports native translation via the decoder prompt
Three ONNX graphs are loaded at runtime:
-
encoder.onnx— FastConformer encoder (~1 GB) -
decoder.onnx— Transformer decoder (~3 GB) -
tokenizer.json+vocab.txt— SentencePiece
The upstream NVIDIA Canary-1B v2 advertises 25 languages. The Rust port in dictee exposes 7 (verified in src/canary.rs:33-47):
| ISO | Language | Canary token ID |
|---|---|---|
en |
English | 64 |
fr |
French | 71 |
de |
German | 78 |
es |
Spanish | 171 |
it |
Italian | 87 |
pt |
Portuguese | 138 |
uk |
Ukrainian | 182 |
Other languages the model could theoretically handle are not wired up on the dictee side — if you need one, open an issue or switch to faster-whisper (99 languages).
Unlike Parakeet (transcribe only, translation delegated to Ollama / Google / LibreTranslate), Canary does both in the same forward pass via the decoder prompt:
<|startoftranscript|><|emo:undefined|><source_lang><target_lang><|pnc|><|noitn|><|notimestamp|><|nodiarize|>
The <source_lang> and <target_lang> tokens tell the decoder:
- Equal → plain transcription
- Different → translation (audio in
source_lang, text output intarget_lang)
Supported translation pairs: each of the 7 languages ↔ EN (12 directed pairs total, 14 non-identity combinations). Non-English-centric pairs (e.g. FR → DE) are not officially tested — the model may produce something but quality is not guaranteed.
CLI example:
# Transcribe FR
dictee-switch-backend asr canary
DICTEE_LANG_SOURCE=fr dictee
# Transcribe FR → translate EN (all in one)
DICTEE_LANG_SOURCE=fr DICTEE_LANG_TARGET=en dictee --translate
# Transcribe EN → translate FR
DICTEE_LANG_SOURCE=en DICTEE_LANG_TARGET=fr dictee --translateNo need for Ollama/LibreTranslate to be running, no need for network access: Canary does it all locally.
Canary expects 16 kHz mono. dictee handles resampling automatically.
Preprocessor parameters (verified in src/canary.rs:171-183):
| Parameter | Value |
|---|---|
| Feature size (mel bins) | 128 |
| n_fft | 512 |
| hop_length | 160 (10 ms @ 16 kHz) |
| win_length | 400 (25 ms @ 16 kHz) |
| Pre-emphasis | 0.97 |
| Window | Hann |
| Padding side | right |
| Padding value | 0.0 |
The full mel-spectrogram is fed to the encoder in one shot — no internal chunking. This is what drives the practical duration limits (next section).
On an RTX 4070 Laptop (8 GB) — real measurements via nvidia-smi:
- Daemon started, model loaded: ~5.3 GB VRAM
- Peak during 5 s utterance: ~5.3 GB (stable)
- Peak during 60 s utterance: ~6.3 GB (+1 GB for attention matrices)
FastConformer encoder self-attention is quadratic in sequence length — peak VRAM grows with audio duration. On an 8 GB 4070 Laptop, practical limits depend on how much VRAM is free at call time:
| Audio length | Peak VRAM estimate | Min free VRAM needed | Outcome |
|---|---|---|---|
| 3-10 s | ~5.3 GB | ~200 MB | Clean |
| 30 s | ~5.3 GB | ~300 MB | Clean (see benches) |
| 60 s | ~6.3 GB | ~1 GB | Truncated output (~780 chars) |
| 300 s (5 min) | silent OOM | — | Empty output (~340 chars) |
Gotcha: if other GPU processes are running (Ollama, a game, a browser with hardware acceleration, etc.) and eat into free VRAM, Canary can slip into a hallucination loop on a duration that would normally work. Observed on a 30 s bench with only ~4.2 GB free at start: 30.4 s latency and output repeated to the token cap (~2 432 chars). Same bench with ~2.5 GB free at minimum during inference (Ollama stopped): 1.55 s latency, clean output.
Recommendations:
- On 8 GB VRAM: Canary is reliable up to 30 s when nothing else is hammering the GPU
- Beyond that: switch to Parakeet (internal chunking, no tangible ceiling) or Whisper
- For long audio files (meetings, podcasts), always use Parakeet — Canary is not designed for it
Canary produces native punctuation (., ,, ?, !) and capitalization (sentence start + proper nouns). The <|pnc|> token is included in the prompt by default — disabling it would require editing the prompt construction (not exposed as user config).
Consequence: the Capitalization step of post-processing does almost nothing on Canary output, unlike Vosk which needs a massive cleanup.
Canary works on CPU (ONNX Runtime provider fallback: CPUExecutionProvider), but latency is unusable:
| Audio length | GPU (RTX 4070) | CPU (i7-13700H) |
|---|---|---|
| 5 s | 0.18 s | ~4.5 s |
| 10 s | 0.36 s | ~9 s |
The multi-layer Transformer decoder with autoregressive generation is 3× slower than Parakeet's TDT decoder on CPU. For interactive use, NVIDIA GPU with 6+ GB VRAM required.
Important difference vs Parakeet: Canary does not auto-detect language. You must set DICTEE_LANG_SOURCE=<code> before speaking (via dictee-setup or the plasmoid language combo). If you speak French with DICTEE_LANG_SOURCE=en, Canary will translate into English instead of transcribing in French — model-correct behavior but disconcerting.
Measured example (WER bench on 20 LibriSpeech EN clips):
-
DICTEE_LANG_SOURCE=fr: WER = 101 % (Canary returns French because source=fr + EN audio → EN→FR translation) -
DICTEE_LANG_SOURCE=en: WER = 1.8 %
If you switch languages frequently, use Parakeet instead (it auto-detects).
After downloading via dictee-setup or the CLI command dictee-setup --download-canary:
/usr/share/dictee/canary/
├── encoder.onnx (~1 GB)
├── decoder.onnx (~3 GB)
├── vocab.txt (~50 KB — SentencePiece tokens)
└── tokenizer.json (~3 MB — HuggingFace tokenizer config)
Or in user-space mode:
~/.local/share/dictee/canary/
└── (same layout)
The dictee-canary.service daemon picks the right path automatically via dynamic resolution.
Measured on a TUXEDO InfinityBook Pro Gen8 (MK2) — Intel Core i7-13700H, RTX 4070 Laptop 8 GB, TUXEDO OS (kernel 6.17, NVIDIA 590.48.01), ONNX Runtime CUDA.
5 runs per duration + 1 discarded warm-up, clips generated by concatenating LibriSpeech dev-clean files. Canary loaded alone (Parakeet stopped) to maximize free VRAM at call time.
| Audio length | Canary (RTX 4070) | Parakeet (RTX 4070) | Canary observation |
|---|---|---|---|
| 3 s | 0.203 s (min 0.200 · max 0.213) | 0.039 s | Clean |
| 5 s | 0.219 s (min 0.219 · max 0.237) | 0.045 s | Clean |
| 10 s | 0.401 s (min 0.396 · max 0.431) | 0.081 s | Clean |
| 30 s | 1.554 s (min 1.538 · max 1.566) | 0.414 s | Clean (721 chars) |
| 60 s | 4.05 s | 0.711 s | Truncated (~780 chars) |
| 300 s (5 min) | 0.26 s | 5.58 s | Empty (~340 chars, silent OOM) |
Parakeet stays linear over the whole range thanks to internal chunking (RTF ~54× real-time even on 5 min). Canary stays reliable up to 30 s then falls off — the encoder needs more VRAM than available to fit full attention.
Note on VRAM sensitivity: an initial bench with only ~4.2 GB free at start (Ollama + Chrome active) showed 30.4 s on the 30 s clip (hallucination loop, 2 432 repeated chars). With Ollama stopped (~2.5 GB free at minimum during inference), the same clip processes cleanly in 1.55 s. On a loaded machine, Canary can slip into degraded mode well before reaching its "true" ceiling.
Raw output, no post-processing:
| Language | Dataset | Canary WER | Canary CER | Parakeet WER | Parakeet CER |
|---|---|---|---|---|---|
| FR | MultiLingual LibriSpeech | 5.4 % | 2.1 % | 7.4 % | 4.0 % |
| EN | LibriSpeech clean | 1.8 % | 0.5 % | 2.0 % | 0.6 % |
Quality distribution (20 FR clips each):
| Canary FR | Parakeet FR | |
|---|---|---|
| Perfect (WER = 0 %) | 8/20 | 4/20 |
| Good (WER < 10 %) | 18/20 | 15/20 |
| Acceptable (< 30 %) | 20/20 | 19/20 |
| Bad (≥ 30 %) | 0/20 | 1/20 |
Verdict: Canary wins on quality for both tested languages, at a 3-5× higher latency on short utterances. For short dictations (< 10 s), Canary is the better pick when you have the VRAM.
- ASR-Backends — compare Canary with Parakeet, Whisper, Vosk
- Translation — compare Canary (built-in translation) with Ollama, Google, LibreTranslate
- Parakeet-TDT-Deep-Dive — the other NeMo backend, with language auto-detection and chunking
- Troubleshooting — VRAM OOM recovery