Skip to content

fix(evals): restore speech when switching from text to audio sessions - #5722

Open
skulitom wants to merge 2 commits into
pipecat-ai:mainfrom
skulitom:fix/eval-tts-per-connection
Open

fix(evals): restore speech when switching from text to audio sessions#5722
skulitom wants to merge 2 commits into
pipecat-ai:mainfrom
skulitom:fix/eval-tts-per-connection

Conversation

@skulitom

Copy link
Copy Markdown

The eval transport now applies skip_tts on every client connection before on_client_connected, so an audio scenario can produce speech after a text-mode scenario on the same bot process. Previously, only truthy flags were sent to the LLM service; the True setting persisted when the next client omitted the flag or sent skip_tts=false.

This addresses the separate eval-harness problem reported in #5425 (comment). The original Flux/muting issue is outside this change.

Validation:

  • Two regression tests exercise text -> audio -> text connections and inspect the active output setting when each connection callback runs. Both fail on main with [True, True, True] instead of [True, False, True], and pass with the fix.
  • pytest tests/test_evals_transport.py tests/test_websocket_server_transport.py tests/test_evals_session.py: 120 passed.
  • Repository-wide ruff check and ruff format --check pass; scoped Pyright reports no errors or warnings.

AI disclosure: OpenAI Codex prepared the code, tests, and PR text and ran the validation above. The new regression tests use mocked transport I/O; no live LLM/TTS provider evaluation was run.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant