fix: address 5 Detail scan bugs from March 11 (reconnect, mutex leak, playout, ordering, retryability)

[toubatbrian]

Bugs Fixed

1) STT reconnect stream could break permanently

• File: agents/src/inference/stt.ts
• Fix: Switched from reusing a shared abort controller to creating a per-connection AbortController in the reconnect loop, while still propagating the parent abort signal.
• Why: Reconnect cleanup was aborting a controller that was then reused, causing subsequent iterations to start already aborted.

2) ProcPool MultiMutex slots could leak

• File: agents/src/ipc/proc_pool.ts
• Fix: Store warmed processes as { proc, unlock } and release the exact lock token associated with the dequeued process; also release queued lock tokens on close and on warmup failure.
• Why: A single shared procUnlock was overwritten in multi-idle scenarios, losing earlier unlock functions.

3) Overlapping audio segments could hang playout flow

• File: agents/src/voice/room_io/_output.ts
• Fix: Snapshot segment duration at flush start and subtract only that accounted amount instead of resetting pushedDuration to zero.
• Why: Resetting to zero could erase duration from a newly overlapping segment and make later flush() calls no-op.

4) LLM context grouping order could become nondeterministic

• File: agents/src/llm/provider_format/utils.ts
• Fix: Assign insertionOrder for the non-assistant branch as well.
• Why: Missing insertion indices produced undefined - undefined in sort and unstable ordering.
• Note: agent_handoff handling remains intentionally filtered from provider model input; comments were clarified.

5) HTTP 408/429 were treated as non-retryable

• File: agents/src/_exceptions.ts
• Fix: Default retryability now treats 408 and 429 as retryable while preserving explicit overrides.
• Why: Both are transient statuses and should retry by default.

Tests Added

• agents/src/_exceptions.test.ts
  • Validates 408/429 retryability defaults and override behavior.
• agents/src/ipc/proc_pool.test.ts
  • Validates correct lock release on dequeue and during pool close.
• agents/src/llm/provider_format/utils.test.ts
  • Validates stable non-assistant insertion ordering.
• agents/src/voice/room_io/_output.test.ts
  • Validates overlapping segment duration is preserved after flush completion.

Validation

pnpm vitest run \
  agents/src/_exceptions.test.ts \
  agents/src/ipc/proc_pool.test.ts \
  agents/src/llm/provider_format/utils.test.ts \
  agents/src/voice/room_io/_output.test.ts \
  agents/src/inference/stt.test.ts \
  agents/src/llm/provider_format/openai.test.ts \
  agents/src/llm/provider_format/google.test.ts

All tests passed.

[changeset-bot]

🦋 Changeset detected

Latest commit: de2a152

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 22 packages

Name	Type
@livekit/agents	Patch
@livekit/agents-plugin-anam	Patch
@livekit/agents-plugin-baseten	Patch
@livekit/agents-plugin-bey	Patch
@livekit/agents-plugin-cartesia	Patch
@livekit/agents-plugin-deepgram	Patch
@livekit/agents-plugin-elevenlabs	Patch
@livekit/agents-plugin-google	Patch
@livekit/agents-plugin-hedra	Patch
@livekit/agents-plugin-inworld	Patch
@livekit/agents-plugin-lemonslice	Patch
@livekit/agents-plugin-livekit	Patch
@livekit/agents-plugin-neuphonic	Patch
@livekit/agents-plugin-openai	Patch
@livekit/agents-plugin-phonic	Patch
@livekit/agents-plugin-resemble	Patch
@livekit/agents-plugin-rime	Patch
@livekit/agents-plugin-sarvam	Patch
@livekit/agents-plugin-silero	Patch
@livekit/agents-plugins-test	Patch
@livekit/agents-plugin-trugen	Patch
@livekit/agents-plugin-xai	Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR