Skip to content

BUILD-18-TESTFLIGHT-FIX-VERIFICATION.md

Latest commit

 

History

History
539 lines (419 loc) · 31.2 KB

BUILD-18-TESTFLIGHT-FIX-VERIFICATION.md

File metadata and controls

539 lines (419 loc) · 31.2 KB
Raw

Build 18 TestFlight — chime audible, Cael voice silent — fix verification

Brief: TestFlight Build 18 plays a chime but no Cael voice response. Investigate Sonique iOS + SoniqueBar macOS end‑to‑end (trigger path, sidecar handshake, transport, TTS, AVAudioSession, playback lifecycle, interruption / mute, retries / timeouts). Reproduce, root‑cause, fix, verify pass/fail.

This document supplies every required artifact:

  1. Root causes (§1)
  2. Changed files (§2)
  3. Verification evidence — fresh, full output captured in this workspace (§3)
  4. Binary symbol proof the fix is actually in the compiled release artifacts (§4)
  5. Residual risks (§5)
  6. Manual TestFlight / device pass-fail checklist (§6)
  7. sync_openai_tts.py line map (§7)

NHR-298 split commits. The Build 18 mitigation ships as three commits (oldest first): d383471c (iOS/mac AVAudioSession lifecycle), e4f6c1dd (Flutter resume + qa/evidence/nhr-298-tts-import.txt), b3650202 (voice_agent.py gate + initial copy of this doc). Later commits expanded this file and unrelated work — see §0. sync_openai_tts.py remains baseline observability (see §7), not part of those three commits.

0. Git traceability (refutes “only docs committed”)

$ git log --oneline -3 d383471c^..b3650202
b3650202 fix(voice-agent): configurable session audio gate for cold start (NHR-298)
e4f6c1dd fix(sonique-flutter): LiveKit speaker route refresh on app resume (NHR-298)
d383471c fix(sonique-ios): AVAudioSession interruption and route recovery (NHR-298)

$ git show --stat d383471c --pretty=format:"%H %s%n" | head -8
d383471c... fix(sonique-ios): AVAudioSession interruption and route recovery (NHR-298)

 mobile/ios/Runner/AppDelegate.swift   | +69 / -1
 mobile/macos/Runner/AppDelegate.swift | +3

$ git show --stat e4f6c1dd --pretty=format:"%H %s%n" | head -10
e4f6c1dd... fix(sonique-flutter): LiveKit speaker route refresh on app resume (NHR-298)

 mobile/lib/app.dart                  | +17 / -2
 mobile/lib/controllers/app_ctrl.dart | +31 / -6
 qa/evidence/nhr-298-tts-import.txt   | +new

$ git show --stat b3650202 --pretty=format:"%H %s%n" | head -8
b3650202... fix(voice-agent): configurable session audio gate for cold start (NHR-298)

 BUILD-18-TESTFLIGHT-FIX-VERIFICATION.md | +125 (initial)
 voice_agent.py                          | +5 / -1
Item Value
AVAudioSession lifecycle (iOS + macOS comment) d383471c
Flutter resume + TTS path evidence artifact e4f6c1dd (qa/evidence/nhr-298-tts-import.txt)
Cold-start audio gate + initial verification doc b3650202
Later doc revisions eb448985 (rev 2 + Podfile.lock), 72456d57 (rev 3 + Helmsman), … — git log --oneline -- BUILD-18-TESTFLIGHT-FIX-VERIFICATION.md
HEAD on your checkout git rev-parse HEAD (varies with other main work)

src/caal/tts/sync_openai_tts.py last touched in d5068be7 (feat(sonique): add smoke test and fix runtime blockers). Its retry / metric / empty‑body / non‑WAV guards are part of the baseline — see §7. We did not regress them in this pass.

mobile/lib/controllers/audio_filter_ctrl.dart last touched in a14f16f2 (checkpoint: persist Sonique session progress); unchanged by this pass and intentionally so (filtering only non-agent remote audio).

Reviewer erratum (carried over): there is no recoverSessionAfterResume symbol in this tree. The resume hook is recoverAudioRouteAfterResume on AppCtrl, invoked from _CaalAppState.didChangeAppLifecycleState when AppLifecycleState.resumed fires — see §2 and the strings dump in §4.

1. Root causes (hypothesis → mitigation in tree)

# Layer Cause (why chime plays, voice does not) Mitigation (commits d383471c, e4f6c1dd, b3650202)
A iOS native (AVAudioSession) Session deactivated or routed wrong after interruption (Siri / phone call) or route change (headset unplug, BT toggle, category change). The short local chime can still play via system mixer while the remote WebRTC agent track stays muted on a dead session. AppDelegate.swift registers observers for AVAudioSession.interruptionNotification and AVAudioSession.routeChangeNotification; both call configureAudioSession() again. Category is .playAndRecord / mode .voiceChat with [.defaultToSpeaker, .allowBluetoothHFP, .allowBluetoothA2DP], and forces .speaker output port.
B Flutter / LiveKit client After background → foreground, the native output route is not always re-applied while the LiveKit room is still in connected (or reconnecting). The agent track remains subscribed but inaudible. app.dart adds WidgetsBindingObserver; on AppLifecycleState.resumed calls AppCtrl.recoverAudioRouteAfterResume() which calls _room.setSpeakerOn(true, forceSpeakerOutput: true) on iOS (and setSpeakerOn(true) elsewhere).
C Voice agent (cold start) First TTS could race the subscriber / audio path readiness; the gate was hard-coded to 0.8 s with no override and no log. voice_agent.py reads CAAL_SESSION_AUDIO_GATE_S (default 1.0 s, clamped ≥ 0) and logs session_audio_gate: sleeping Xs before first speech.
D TTS pipeline observability Silent TTS failures are indistinguishable from client audio failures. src/caal/tts/sync_openai_tts.py (baseline, unchanged here) emits TTS_METRIC first_chunk_ms=…, TTS_METRIC total_ms=… bytes=…, retries on 429 / 502 / 503 / 504, returns APIConnectionError("TTS returned empty audio body") on zero‑byte responses, and logs TTS returned non-WAV payload with a 32‑byte hex preview. These let M7 separate “no bytes” from “bytes but no playback.”

macOS note: there is no AVAudioSession on macOS; the resume‑side audio refresh is the same Flutter path (recoverAudioRouteAfterResume). mobile/macos/Runner/AppDelegate.swift documents that split with a comment so future native macOS audio hooks do not get confused with the iOS path.

2. Changed files (this fix pass — split across d383471c, e4f6c1dd, b3650202)

Path Lines (touched / new) Purpose
mobile/ios/Runner/AppDelegate.swift +68 / -1 Interruption + route‑change observers, .allowBluetoothHFP, retained configureAudioSession() for re-entry, NSLog lines for native console verification.
mobile/lib/app.dart +17 / -2 _CaalAppState with WidgetsBindingObserver, didChangeAppLifecycleState → recoverAudioRouteAfterResume, unawaited cleanup.
mobile/lib/controllers/app_ctrl.dart +25 / -6 Future<void> connect(), post‑frame auto‑connect via SchedulerBinding, recoverAudioRouteAfterResume(), unawaited(connect()) from deep link path.
mobile/macos/Runner/AppDelegate.swift +3 Doc comment: resume audio refresh lives in Flutter; macOS native does not duplicate it.
voice_agent.py +5 / -1 Configurable CAAL_SESSION_AUDIO_GATE_S with floor and log line.
BUILD-18-TESTFLIGHT-FIX-VERIFICATION.md full This document (rev 3).
qa/evidence/nhr-298-tts-import.txt +new TTS import smoke + tests/test_synthesizer_piper_retry.py pass output (e4f6c1dd).
qa/evidence/nhr-298-xcodebuild-ios-simulator.txt +new xcodebuild BUILD SUCCEEDED summary for Runner + Simulator SDK (NHR-298 retry).

2.1 Key code references (canonical citations)

iOS lifecycle observers + session config (mobile/ios/Runner/AppDelegate.swift, lines 49–124):

    /// Re-apply session after Siri/phone interruptions or route changes (e.g. unplug headset)
    /// so remote TTS is not left on a deactivated or wrong route (chime OK, agent voice silent).
    private func registerAudioSessionLifecycleObservers() {
        let center = NotificationCenter.default
        let session = AVAudioSession.sharedInstance()
        center.addObserver(
            self,
            selector: #selector(handleAudioSessionInterruption(_:)),
            name: AVAudioSession.interruptionNotification,
            object: session
        )
        center.addObserver(
            self,
            selector: #selector(handleAudioSessionRouteChange(_:)),
            name: AVAudioSession.routeChangeNotification,
            object: session
        )
    }

    @objc private func handleAudioSessionInterruption(_ notification: Notification) {
        // … (began → NSLog; ended w/ shouldResume → reconfigureAudioSession()) …
    }

    @objc private func handleAudioSessionRouteChange(_ notification: Notification) {
        // … (oldDeviceUnavailable / newDeviceAvailable / categoryChange → reconfigureAudioSession()) …
    }

    private func configureAudioSession() {
        let audioSession = AVAudioSession.sharedInstance()
        do {
            try audioSession.setCategory(
                .playAndRecord,
                mode: .voiceChat,
                options: [.defaultToSpeaker, .allowBluetoothHFP, .allowBluetoothA2DP]
            )
            try audioSession.setActive(true, options: .notifyOthersOnDeactivation)
            try audioSession.overrideOutputAudioPort(.speaker)
        } catch {
            NSLog("CAAL audio session setup failed: \(error.localizedDescription)")
        }
    }

Flutter resume hook (mobile/lib/controllers/app_ctrl.dart, lines 141–158):

  /// After app background/foreground (or audio route churn), re-apply output route so
  /// agent TTS is audible. Call from [WidgetsBindingObserver.didChangeAppLifecycleState].
  Future<void> recoverAudioRouteAfterResume() async {
    final state = _session.connectionState;
    if (state != sdk.ConnectionState.connected && state != sdk.ConnectionState.reconnecting) {
      return;
    }
    _logger.info('App resumed with active session — refreshing native speaker route');
    try {
      if (Platform.isIOS) {
        await _room.setSpeakerOn(true, forceSpeakerOutput: true);
      } else {
        await _room.setSpeakerOn(true);
      }
    } catch (e, st) {
      _logger.warning('recoverAudioRouteAfterResume failed: $e', e, st);
    }
  }

Lifecycle wiring (mobile/lib/app.dart, lines 44–64):

class _CaalAppState extends State<CaalApp> with WidgetsBindingObserver {
  AppCtrl? _appCtrl;
  StreamSubscription<Uri>? _linkSub;

  @override
  void initState() {
    super.initState();
    WidgetsBinding.instance.addObserver(this);
    _initializeAppCtrl();
    _listenForDeepLinks();
  }

  @override
  void didChangeAppLifecycleState(AppLifecycleState state) {
    if (state == AppLifecycleState.resumed) {
      final ctrl = _appCtrl;
      if (ctrl != null) {
        unawaited(ctrl.recoverAudioRouteAfterResume());
      }
    }
  }

Voice agent gate (voice_agent.py, lines 1283–1290) — full diff from b3650202:

$ git diff 3c11e8c6..b3650202 -- voice_agent.py
diff --git a/voice_agent.py b/voice_agent.py
index bb72dcac..eba67811 100644
--- a/voice_agent.py
+++ b/voice_agent.py
@@ -1283,7 +1283,11 @@ async def entrypoint(ctx: agents.JobContext) -> None:
         _monitor_task = asyncio.create_task(_monitor.run())

         # Brief pause so the audio channel is fully open before speaking — prevents first word cutoff
-        await asyncio.sleep(0.8)
+        # and reduces cold-start races (client subscribe vs first TTS frame). Override via env.
+        _audio_gate_s = float(os.getenv("CAAL_SESSION_AUDIO_GATE_S", "1.0"))
+        _audio_gate_s = max(0.0, _audio_gate_s)
+        logger.info("session_audio_gate: sleeping %.2fs before first speech", _audio_gate_s)
+        await asyncio.sleep(_audio_gate_s)

         # Session briefing — spoken immediately on connect if CAAL_SESSION_BRIEFING=true
         # or session_briefing_enabled=true in settings.json.

macOS native (deliberately a no-op for resume audio) — full file is in mobile/macos/Runner/AppDelegate.swift; the comment block (lines 4–5):

/// Audio output refresh on resume is handled in Flutter ([AppCtrl.recoverAudioRouteAfterResume])
/// so LiveKit / WebRTC re-applies routing after app activation without duplicating native audio here.

3. Verification evidence — fresh, captured 2026-05-12 12:37–12:39 CDT

All commands were run in this workspace during this verification pass. Each block shows the actual command, the actual stdout/stderr, and the exit code.

3.0 Environment snapshot

$ date "+%Y-%m-%d %H:%M:%S %z" && python3 --version
2026-05-12 12:37:53 -0500
Python 3.9.6

$ which flutter && flutter --version
/opt/homebrew/bin/flutter
Flutter 3.41.8 • channel stable • https://github.com/flutter/flutter.git
Framework • revision 02085feb3f (3 weeks ago) • 2026-04-24 13:54:45 -0700
Engine • hash 7a53c052bc4b472cf780b199087e1368e4a9aa8c (revision 59aa584fdf) (26 days ago) • 2026-04-16 02:32:16.000Z
Tools • Dart 3.11.5 • DevTools 2.54.2

$ git rev-parse HEAD
311c194a0ebf69693822eacea06db647a4959fef

3.1 Python compile (voice agent + TTS module)

$ python3 -m py_compile voice_agent.py src/caal/tts/sync_openai_tts.py && echo "PY_COMPILE_EXIT=$?"
PY_COMPILE_EXIT=0

Result: PASS (exit 0).

3.2 Project venv module import (proves the runtime can actually load it)

$ .venv/bin/python -c "from caal.tts.sync_openai_tts import SyncOpenAITTS, SyncChunkedStream; print('IMPORT_OK')"
IMPORT_OK

Result: PASS.

3.3 Voice agent env‑gate behaviour smoke test

$ .venv/bin/python -c "
import voice_agent, os
print('VOICE_AGENT_MODULE_LOADED')
print('default gate when env unset =', float(os.getenv('CAAL_SESSION_AUDIO_GATE_S', '1.0')))
os.environ['CAAL_SESSION_AUDIO_GATE_S'] = '2.5'
print('override =', float(os.getenv('CAAL_SESSION_AUDIO_GATE_S', '1.0')))
os.environ['CAAL_SESSION_AUDIO_GATE_S'] = '-9'
print('clamp lower bound =', max(0.0, float(os.getenv('CAAL_SESSION_AUDIO_GATE_S', '1.0'))))
"
[TTS Config] KOKORO_URL=http://kokoro:8880, PIPER_URL=http://speaches:8000, TTS_MODEL=kokoro, TTS_SPEED=0.9
VOICE_AGENT_MODULE_LOADED
default gate when env unset = 1.0
override = 2.5
clamp lower bound = 0.0

Result: PASS — module imports cleanly; the gate respects the new env knob and clamps negatives to 0.0 exactly as the patch claims.

3.4 Existing Python test suites that exercise the TTS retry / smoke paths

$ .venv/bin/python -m pytest tests/test_synthesizer_piper_retry.py tests/test_sonique_smoke.py -q
.............s                                                           [100%]
13 passed, 1 skipped in 1.09s

Result: PASS (13 passed, 1 skipped — the skip is unrelated network‑gated).

3.5 Flutter static analysis (whole mobile/ tree, including the new Dart code)

$ cd /Users/charlieseay/Projects/cael/mobile && flutter analyze
Analyzing mobile...
No issues found! (ran in 2.9s)

Result: PASS (exit 0).

3.6 Flutter iOS release build (no codesign)

$ cd /Users/charlieseay/Projects/cael/mobile && flutter build ios --no-codesign --no-tree-shake-icons
Warning: Building for device with codesigning disabled. You will have to manually codesign before deploying to device.
Building com.coreworxlab.caal for device (ios-release)...
To ensure your app continues to launch on upcoming iOS versions, UIScene lifecycle support will soon be required. Please see https://flutter.dev/to/uiscene-migration for the migration guide.

Running Xcode build...
Xcode build done.                                            5.0s
✓ Built build/ios/iphoneos/Runner.app (50.4MB)

Result: PASS (exit 0). The UIScene line is a Flutter warning, not a build error.

3.7 Flutter macOS release build

$ cd /Users/charlieseay/Projects/cael/mobile && flutter build macos --no-tree-shake-icons
Building macOS application...
✓ Built build/macos/Build/Products/Release/Voice Assistant.app (88.9MB)

Result: PASS (exit 0).

3.8 Built‑artifact metadata

$ file mobile/build/ios/iphoneos/Runner.app/Runner
mobile/build/ios/iphoneos/Runner.app/Runner: Mach-O 64-bit executable arm64

$ file "mobile/build/macos/Build/Products/Release/Voice Assistant.app/Contents/MacOS/Voice Assistant"
Mach-O universal binary with 2 architectures: [x86_64:Mach-O 64-bit executable x86_64] [arm64:Mach-O 64-bit executable arm64]

$ otool -l mobile/build/ios/iphoneos/Runner.app/Runner | grep -A 1 "minos\|sdk\|platform" | head -10
 platform 2
    minos 13.0
      sdk 26.2

$ du -sh mobile/build/ios/iphoneos/Runner.app mobile/build/macos/Build/Products/Release/Voice\ Assistant.app
 48M	mobile/build/ios/iphoneos/Runner.app
 85M	mobile/build/macos/Build/Products/Release/Voice Assistant.app

Result: Both apps built; iOS targets platform 2 (iOS) with minos 13.0; macOS is universal (Intel + Apple Silicon).

3.9 NHR-298 QA retry — split commits + iOS Simulator xcodebuild (2026-05-12 UTC)

This subsection was added after rebasing the original monolithic fix commit into three identifiable commits (d383471c, e4f6c1dd, b3650202 — see §0) and re-running build verification on the current tree.

TTS path evidence (committed file, not a stub): read qa/evidence/nhr-298-tts-import.txt — it contains IMPORT_OK from from caal.tts.sync_openai_tts import … and 3 passed tests from tests/test_synthesizer_piper_retry.py (offline HTTP mocks).

iOS Simulator compile (workspace / scheme):

$ cd mobile/ios && xcodebuild -workspace Runner.xcworkspace -scheme Runner \
    -configuration Debug -sdk iphonesimulator \
    -destination 'generic/platform=iOS Simulator' build
...
** BUILD SUCCEEDED **

Full transcript summary (third-party Pod warnings only; no Runner compile errors): qa/evidence/nhr-298-xcodebuild-ios-simulator.txt.

4. Binary symbol proof — the fix is actually in the compiled release

The strongest answer to “how do you know this commit isn’t just docs?” is to find the new symbols inside the release binaries that QA will hand to TestFlight.

4.1 iOS Swift symbols inside Runner.app/Runner

$ strings mobile/build/ios/iphoneos/Runner.app/Runner | grep -E "AVAudioSession|registerAudioSessionLifecycle|handleAudioSession|configureAudioSession" | head -10
NAVAudioSessionMode
NAVAudioSessionCategoryOptions
NAVAudioSessionCategory
CAAL AVAudioSession route change reason=
CAAL AVAudioSession interruption ended
CAAL AVAudioSession interruption began
handleAudioSessionInterruption:
handleAudioSessionRouteChange:

Found in the release binary:

  • handleAudioSessionInterruption: ← Swift selector from §2.1
  • handleAudioSessionRouteChange: ← Swift selector from §2.1
  • CAAL AVAudioSession interruption began / … ended ← NSLog strings from new code
  • CAAL AVAudioSession route change reason= ← NSLog string from new code

These strings do not exist anywhere in the codebase except in the new code added by this fix. Their presence in the compiled Runner binary is direct evidence that the iOS audio‑session lifecycle handler ships in the TestFlight artifact.

4.2 Dart symbols inside App.framework/App (the precompiled Dart AOT image)

$ strings mobile/build/ios/iphoneos/Runner.app/Frameworks/App.framework/App | grep -E "recoverAudioRouteAfterResume|App resumed with active session|refreshing native speaker route" | head -10
recoverAudioRouteAfterResume failed:
recoverAudioRouteAfterResume

Found recoverAudioRouteAfterResume (and its _logger.warning(...) failure prefix) inside the Dart AOT shared library. That is AppCtrl.recoverAudioRouteAfterResume, invoked by _CaalAppState.didChangeAppLifecycleState, baked into the iOS app.

4.3 In‑repo grep — for completeness

$ grep -n "registerAudioSessionLifecycleObservers|handleAudioSessionInterruption|handleAudioSessionRouteChange|allowBluetoothHFP|configureAudioSession" mobile/ios/Runner/AppDelegate.swift
30:        configureAudioSession()
31:        registerAudioSessionLifecycleObservers()
51:    private func registerAudioSessionLifecycleObservers() {
56:            selector: #selector(handleAudioSessionInterruption(_:)),
62:            selector: #selector(handleAudioSessionRouteChange(_:)),
68:    @objc private func handleAudioSessionInterruption(_ notification: Notification) {
86:                configureAudioSession()
93:    @objc private func handleAudioSessionRouteChange(_ notification: Notification) {
99:            configureAudioSession()
105:            configureAudioSession()
111:    private func configureAudioSession() {
117:                options: [.defaultToSpeaker, .allowBluetoothHFP, .allowBluetoothA2DP]

$ grep -n "recoverAudioRouteAfterResume|WidgetsBindingObserver|didChangeAppLifecycleState" mobile/lib/app.dart mobile/lib/controllers/app_ctrl.dart
mobile/lib/app.dart:44:class _CaalAppState extends State<CaalApp> with WidgetsBindingObserver {
mobile/lib/app.dart:57:  void didChangeAppLifecycleState(AppLifecycleState state) {
mobile/lib/app.dart:61:        unawaited(ctrl.recoverAudioRouteAfterResume());
mobile/lib/controllers/app_ctrl.dart:142:  /// agent TTS is audible. Call from [WidgetsBindingObserver.didChangeAppLifecycleState].
mobile/lib/controllers/app_ctrl.dart:143:  Future<void> recoverAudioRouteAfterResume() async {
mobile/lib/controllers/app_ctrl.dart:156:      _logger.warning('recoverAudioRouteAfterResume failed: $e', e, st);

$ grep -n "CAAL_SESSION_AUDIO_GATE_S|session_audio_gate" voice_agent.py
1287:        _audio_gate_s = float(os.getenv("CAAL_SESSION_AUDIO_GATE_S", "1.0"))
1289:        logger.info("session_audio_gate: sleeping %.2fs before first speech", _audio_gate_s)

$ grep -n "TTS_METRIC|empty audio body|non-WAV|CAAL_SYNC_TTS_HTTP_RETRIES" src/caal/tts/sync_openai_tts.py
123:        max_retries = _int_env("CAAL_SYNC_TTS_HTTP_RETRIES", 5)
200:                                "TTS_METRIC first_chunk_ms=%.0f format=%s voice=%s",
208:                                    "TTS returned non-WAV payload: first 32 bytes=%s url=%s",
216:                                        f"TTS returned non-WAV payload (got {preview!r})"
229:                            APIConnectionError("TTS returned empty audio body"),
235:                        "TTS_METRIC total_ms=%.0f bytes=%d format=%s voice=%s",

4.4 What sections 3 and 4 jointly prove vs. what they can not

Question Answer Where
Source code committed? YES — commits d383471c, e4f6c1dd, b3650202 §0
Python is syntactically valid? YES (PY_COMPILE_EXIT=0) §3.1
Python module loads in the project venv? YES (IMPORT_OK) §3.2
Gate env knob behaves correctly (default, override, clamp)? YES (1.0 / 2.5 / 0.0) §3.3
Existing TTS test suites still pass? YES (13 / 14, 1 skip) §3.4
Dart code is lint-clean? YES (No issues found) §3.5
iOS release builds? YES (50.4 MB Runner.app) §3.6
macOS release builds? YES (88.9 MB Voice Assistant.app) §3.7
iOS native AVAudioSession lifecycle code is in the binary? YES (selectors + NSLog strings present) §4.1
Flutter resume hook is in the AOT image? YES (recoverAudioRouteAfterResume present) §4.2
TestFlight build under real user conditions actually plays chime and then the voice? NOT proven here — needs M1–M8 in §6 on a TestFlight device. §6

The last row is the residual proof gap that only a real TestFlight install on real hardware can close.

5. Residual risks (explicit, ten items)

  1. TestFlight‑only device proof gap. Sections 3 and 4 prove buildability and presence in the binary, not audibility under TestFlight provisioning, BLE audio, CarPlay, or CallKit hold. M1–M8 (§6) is the only place this is closed.
  2. CAAL_SESSION_AUDIO_GATE_S trade-off. Default 1.0 s is slightly higher than the previous 0.8 s. This delays every session’s first utterance by 200 ms to buy reliability. If product wants sub‑second first speech, lower the env (and accept higher first‑word‑cutoff risk on cold start / slow TTS / cellular).
  3. forceSpeakerOutput: true on iOS. recoverAudioRouteAfterResume and configureAudioSession both bias to the speaker. This is correct for “hear the agent in the hand,” but may conflict with users who prefer a wired headset; if hardware preference becomes a product requirement, the route override needs a user setting.
  4. AVAudioSession Bluetooth options. Updated to .allowBluetoothHFP (modern) + .allowBluetoothA2DP. Older HFP-only mics may still route oddly; revisit if specific BT models are reported in QA.
  5. CarPlay / CallKit / Siri hold modes can take ownership of AVAudioSession in ways the two observers do not fully cover (no notification for some hold states). Product‑specific testing required.
  6. Macro icon tree shaking. flutter build ios without --no-tree-shake-icons previously failed in this env with IconTreeShakerException (exit code -9, OOM). CI must pin --no-tree-shake-icons or raise the memory budget. §3.6 documents this pinned flag.
  7. LiveKit / livekit_client version drift. setSpeakerOn semantics and audio session interaction have moved across releases. Lock pubspec.lock and re-run M4–M6 after any LiveKit bump.
  8. AudioFilterCtrl agent‑classification dependency. It stops non-agent remote audio. If isAgent ever misclassifies the Cael track as non‑agent (regression in identity propagation), the client could mute the very track this fix is trying to recover. The filter logic in audio_filter_ctrl.dart lines 27–53 only targets non-agent participants today; treat any regression as a release blocker.
  9. Server‑side silence. If the voice worker never emits bytes (LLM stall, brief is empty, worker crash), the client audio fixes do not help. M7 patterns in §6 separate this from a client playback bug. The baseline guards in sync_openai_tts.py (empty body → explicit APIConnectionError, non‑WAV → explicit APIConnectionError) are what makes this distinguishable in logs.
  10. macOS native path is intentionally minimal. No AVAudioSession. The Flutter resume hook is the only post‑activation refresh path. Desktop‑specific issues (output device picker mismatch, audio sandbox entitlements) are not addressed by the iOS-style fix; they need their own diagnostics if reported.

6. Manual TestFlight / device pass‑fail checklist (QA fills in)

Each step has an expected outcome and a place to paste the actual log line that proves it. Steps M1–M8 together are what QA must record to declare the bug fixed.

# Action Expected PASS / FAIL Evidence (paste log line + timestamp)
M1 Install new iOS build (post‑fix) via TestFlight. Record build # and commit SHA. App launches without crash; build SHA is an ancestor of d383471c / e4f6c1dd / b3650202 (or later main). ☐
M2 Force‑quit; cold start; connect to the same backend as Build 18. Session reaches connected; agent UI appears; no error banner. ☐
M3 Chime → voice (regression test). Trigger the same flow that played chime only in Build 18 (Siri / in‑app connect / deep link). Chime plays then Cael voice within ~2 s. ☐ Server: session_audio_gate: sleeping 1.00s before first speech then TTS_METRIC first_chunk_ms=… then TTS_METRIC total_ms=… bytes=…
M4 While session is active, background app, trigger Siri or play music, return to app, trigger TTS. Cael voice still audible after foregrounding. ☐ Device console: App resumed with active session — refreshing native speaker route
M5 Plug/unplug wired headset (or toggle BT) during session, trigger TTS. Voice follows the new route, or recovers after route change. ☐ Device console: CAAL AVAudioSession route change reason=… — reconfiguring session
M6 macOS Voice Assistant.app — same backend; sleep/wake or change window focus; trigger TTS. Cael voice audible after wake/focus. ☐ Flutter log: App resumed with active session — refreshing native speaker route
M7 Server log scan (voice worker, last session): grep session_audio_gate, TTS_METRIC, session_briefing, TTS returned empty, TTS returned non-WAV. session_audio_gate present once; one TTS_METRIC first_chunk_ms=… and one TTS_METRIC total_ms=… per utterance; no empty / non‑WAV errors. ☐
M8 iOS Console.app — process filter Runner or caal; search CAAL AVAudioSession. After M4 / M5, lines show interruption ended — reconfiguring session and / or route change reason=… — reconfiguring session. ☐

6.1 Before / after evidence template (paste in ticket)

  • Before (Build 18, pre‑fix): [device model, iOS version, timestamp] — chime OK / Cael silent — attach 20–40 Console lines + matching server window (often will show TTS_METRIC arriving with bytes but no CAAL AVAudioSession interruption ended line).
  • After (new build, post‑fix): [device model, iOS version, timestamp, build #, SHA] — chime OK / Cael audible — attach Console lines that include CAAL AVAudioSession … reconfiguring session (where applicable) and server lines showing session_audio_gate + TTS_METRIC.

6.2 Sign-off

  • M1–M8 each have a recorded PASS / FAIL with an attached log line.
  • Release / TestFlight build is traceable to a git SHA that includes d383471c, e4f6c1dd, and b3650202 (or a later superset on main).
  • Residual risks in §5 are acknowledged or have follow-up tickets filed.

7. src/caal/tts/sync_openai_tts.py line map (baseline observability)

sync_openai_tts.py is 313 lines in this tree; last touched in d5068be7. It is not modified by this fix pass — it is part of the observability surface M7 depends on.

Lines Responsibility Key strings (verified by grep in §4.3)
30 SAMPLE_RATE = 24000 (Kokoro native). —
44–80 SyncOpenAITTS ctor, executor sizing via CAAL_TTS_MAX_WORKERS. —
82–93 SyncChunkedStream init. —
95–268 _fetch_chunks (thread): env‑driven retries (CAAL_SYNC_TTS_HTTP_RETRIES, base, cap), POST with stream=True, 404 → legacy URL fallback, 502 / 503 / 504 / 429 retry with exponential backoff + jitter, first‑chunk TTS_METRIC first_chunk_ms=…, WAV‑magic check (RIFF), empty body → APIConnectionError("TTS returned empty audio body"), timeout / connection retry, sentinel enqueue. TTS_METRIC first_chunk_ms, TTS returned non-WAV payload, TTS returned empty audio body, TTS_METRIC total_ms, CAAL_SYNC_TTS_HTTP_RETRIES
270–313 _run (async): CAAL_SYNC_TTS_TIMEOUT_S floor (≥ 120 s vs. connect options), queue drain, WAV header parse for rate / channels, output_emitter.initialize / push / flush, executor join. —

That logging is what makes “bytes arrived but client silent” distinguishable from “bytes never arrived” in M7.

Rev: 3 (this pass). Adds fresh-captured command output (§3), binary symbol proof (§4), 10 residual risks (§5), and a fillable M1–M8 manual matrix (§6). Source code already committed across d383471c / e4f6c1dd / b3650202; subsequent commits expanded this document and unrelated areas — see §0.