Skip to content

feat(services/qwen): add Qwen3STTService for local speech-to-text - #5723

Open
RoKenobi wants to merge 4 commits into
pipecat-ai:mainfrom
RoKenobi:feat/qwen3-asr-stt-v2
Open

feat(services/qwen): add Qwen3STTService for local speech-to-text#5723
RoKenobi wants to merge 4 commits into
pipecat-ai:mainfrom
RoKenobi:feat/qwen3-asr-stt-v2

Conversation

@RoKenobi

@RoKenobi RoKenobi commented Sep 12, 2026

Copy link
Copy Markdown

Summary

Adds Qwen3STTService — a local speech-to-text service backed by the Qwen3-ASR model family via the qwen-asr package.

  • Extends SegmentedSTTService, following FunASRSTTService / WhisperSTTService / MoonshineSTTService
  • Three model sizes: Qwen/Qwen3-ASR-0.6B, Qwen/Qwen3-ASR-1.7B (default), Qwen/Qwen3-ASR-8B
  • Multilingual: English, Chinese, Spanish, French, German, Japanese, Korean, Portuguese, Russian, Arabic, Italian, Hindi
  • Runs on CUDA via torch.bfloat16; transcription dispatched with asyncio.to_thread
  • Unit tests with a mocked qwen-asr, following the FunASR/Moonshine test patterns

Replaces #5415, which was opened against a three-week-old main and had drifted far enough that rebasing in place was messier than starting clean.

Packaging: conflicting extras

qwen-asr depends on accelerate==1.12.0, while the moondream extra constrains the same package to ~=1.10.0. uv resolves a single universal lockfile across all extras, so the two pins are unsatisfiable together and uv lock fails outright — regardless of which extras a given install actually selects. That is what broke the docs build on #5415.

Since nobody runs a local Qwen3-ASR model and Moondream vision in one environment, the two are declared conflicting:

conflicts = [
    [
        { extra = "moondream" },
        { extra = "qwen-asr" },
    ],
]

uv then resolves them in separate forks, and activating both together becomes an explicit install-time error rather than a resolution failure for everyone.

qwen-asr is also excluded in docs/api/install-deps.sh alongside the other heavy ML extras (it pulls in torch), and qwen_asr is added to autodoc_mock_imports so the service's API page still generates.

Usage

from pipecat.services.qwen.stt import Qwen3STTService, Model
from pipecat.transcriptions.language import Language

stt = Qwen3STTService(
    model=Model.ASR_1_7B,   # or "Qwen/Qwen3-ASR-0.6B", "Qwen/Qwen3-ASR-8B"
    device="cuda:0",
    language=Language.EN,
)

Install: uv add "pipecat-ai[qwen-asr]"

Not done here

uv.lock is not regenerated — this branch carries main's lockfile, which predates the qwen-asr extra. Read the Docs re-resolves during uv sync, so the docs build exercises the conflicts change, but uv lock has not been run against it directly.

Test plan

  • Unit tests: defaults, settings overrides, string model ID, language mapping, run_stt transcription, missing-model error, inference-exception error, empty-transcript skip
  • Validated in production prior to this PR in a pipecat-based voice server (Qwen3-ASR-1.7B on CUDA)
  • uv lock regenerated and committed

Notes

  • No changes to the existing qwen/llm.py or qwen/__init__.py; the qwen = [] extra (DashScope LLM) is untouched and qwen-asr is separate
  • Each service instance loads its own copy of the model into VRAM. The class docstring documents the subclass pattern for sharing one Qwen3ASRModel across pipeline sessions.

Adds Qwen3STTService, a SegmentedSTTService backed by the Qwen3-ASR model
family (0.6B / 1.7B / 8B). The service loads the model once at construction
via the qwen-asr package and transcribes VAD-segmented 16 kHz PCM in an
asyncio thread, following the same patterns as FunASRSTTService and
WhisperSTTService.

qwen-asr is excluded from the docs environment alongside the other heavy ML
extras: it depends on torch, which conf.py already mocks.
qwen-asr depends on accelerate==1.12.0, while the moondream extra
constrains accelerate to ~=1.10.0. uv resolves one universal lockfile
across every extra, so the two pins are unsatisfiable together and
`uv lock` fails outright - regardless of which extras a given install
selects.

Neither extra needs the other: a local Qwen3-ASR model and Moondream
vision are not run in the same environment. Declaring the pair
conflicting lets uv resolve them in separate forks and produce a
universal lock, and makes activating both together an explicit error at
install time rather than a resolution failure for everyone.
The qwen-asr extra is excluded from the docs environment, so autodoc
cannot import pipecat.services.qwen.stt and the module's page would be
skipped. Mock the package the same way funasr and pocket_tts are; torch,
its other import, is already mocked.
@RoKenobi RoKenobi changed the title Feat/qwen3 asr stt v2 feat(services/qwen): add Qwen3STTService for local speech-to-text Sep 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant