Skip to content

Speechmatics Agent STT in 1.10.0 inherits unsupported 8 kHz input; sample_rate=16000 relabels PCM without resampling #5717

Description

@mannyb223

Pipecat 1.10.0 migrated SpeechmaticsSTTService to speechmatics-agent-stt 0.1.0. Agent STT requires 16 kHz PCM, but SpeechmaticsSTTService still inherits the pipeline input rate when its constructor sample_rate is omitted. A standard 8 kHz FastAPI websocket pipeline therefore opens /v2/agent with audio_format.sample_rate=8000, and the provider rejects the session. This matters when the service is a backup: the backup may be permanently unusable before it is ever selected.

Environment: pipecat-ai 1.10.0, speechmatics-agent-stt 0.1.0, speechmatics-rt 1.1.1, Python 3.13. The session rejection was observed on Linux. The stock input/SDK byte-path controls described below were run offline on macOS with the same package versions. No client data or production-specific pipeline is needed.

The provider rejection is:

Speechmatics STT rejected the session: invalid input: (root): Must validate one and only one schema (oneOf),audio_format.sample_rate: audio_format.sample_rate does not match: 16000

The mechanism is visible in versioned source. SpeechmaticsSTTService.init accepts sample_rate=None and passes it to STTService. STTService.setup chooses the explicit rate or setup.audio_in_sample_rate. SpeechmaticsSTTService._open_connection passes self.sample_rate into the Agent client's AudioFormat. STTService.process_audio_frame passes frame.audio unchanged to run_stt; SpeechmaticsSTTService.run_stt sends those bytes to the SDK. The SDK's agent_stt/_client.py:251 delegates to rt/_base_client.py:123, which sends the same bytes. None of those paths resamples audio.

Consequently, setting only SpeechmaticsSTTService(sample_rate=16000) is not a repair for an 8 kHz pipeline. It changes the declared rate without changing PCM. The Agent STT 0.1.0 documentation requires actual 16 kHz input. Pipecat's general STT guide recommends consistent pipeline-wide rates, which is a supported configuration direction when every input consumer is migrated. The Speechmatics page still says an omitted rate inherits the pipeline and describes the old endpoint/turn modes, without explaining this new restriction. This is an incompatible-default/validation and migration-documentation report, not a claim that 16 kHz pipelines cannot use Speechmatics.

Minimal stock reproduction: save the following as speechmatics_8k.py. Install pipecat-ai[websocket,speechmatics]==1.10.0 and uvicorn in a disposable environment, set your own SPEECHMATICS_API_KEY, and run python -m uvicorn speechmatics_8k:app --host 127.0.0.1 --port 8765. A websocket connection alone starts the Speechmatics session and exposes the rejected configuration; no caller audio is needed for the rejection. This server recipe is supplied for maintainers; no provider request was made while preparing this issue draft.

import os
from fastapi import FastAPI, WebSocket
from pipecat.pipeline.pipeline import Pipeline
from pipecat.pipeline.worker import PipelineParams, PipelineWorker
from pipecat.serializers.protobuf import ProtobufFrameSerializer
from pipecat.services.speechmatics.stt import SpeechmaticsSTTService
from pipecat.transports.websocket.fastapi import (
    FastAPIWebsocketParams, FastAPIWebsocketTransport,
)
from pipecat.workers.runner import WorkerRunner

app = FastAPI()

@app.websocket("/ws")
async def websocket_endpoint(websocket: WebSocket):
    await websocket.accept()
    transport = FastAPIWebsocketTransport(websocket, FastAPIWebsocketParams(
        audio_in_enabled=True,
        audio_in_sample_rate=8000,
        serializer=ProtobufFrameSerializer(),
    ))
    stt = SpeechmaticsSTTService(api_key=os.environ["SPEECHMATICS_API_KEY"])
    worker = PipelineWorker(
        Pipeline([transport.input(), stt]),
        params=PipelineParams(audio_in_sample_rate=8000),
        enable_rtvi=False,
        enable_turn_tracking=False,
    )
    @transport.event_handler("on_client_disconnected")
    async def disconnected(_transport, _websocket):
        await worker.cancel()
    runner = WorkerRunner(handle_sigint=False)
    await runner.add_workers(worker)
    await runner.run()

Connect a local websocket client:

import asyncio
from websockets.asyncio.client import connect

async def main():
    async with connect("ws://127.0.0.1:8765/ws"):
        await asyncio.sleep(3)

asyncio.run(main())

Expected: validate the service's supported input rate before trying an invalid provider session, clearly document how an 8 kHz transport must deliver genuine 16 kHz pipeline audio, and prevent a service-only override from being mistaken for a converter. If different service rates on one pipeline are supported, provide and document the conversion boundary. A plain sample_rate=16000 override must not silently transmit unconverted 8 kHz PCM as 16 kHz.

Offline controls used a real PipelineWorker, FastAPI input transport and ProtobufFrameSerializer, followed by the real Speechmatics service and real Agent/RT SDK send_audio implementations. Only websocket input, provider connection acceptance/disconnection and the SDK's network send boundary were replaced. The connection stub deliberately accepted all declared rates so the test could observe bytes beyond the rejected configuration. One second of mono signed-16-bit PCM produced: pipeline 8000/service default → declared 8000, 16000 bytes sent; pipeline 8000/service override16000 → declared16000, the same16000 bytes sent; pipeline16000/service default → declared16000, 32000 bytes sent. Incoming frames and final SDK payloads were byte-identical in each case. This proves the absence of conversion, not live acceptance of the control. The provider restriction above comes from its documented requirement and observed rejection.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions