fix(openai realtime): treat create_response=False as client-side turn taking

[longcw]

Server VAD with create_response=False still commits and transcribes the user audio, but the reply is the client's call, so the session no longer reports server-side turn detection and allow_interruptions=False is honored.

Fixes #6635

[devin-ai-integration]

Devin Review found 1 new potential issue.

View 3 additional findings in Devin Review.

[biztex]

Treating create_response=False as client-side turn taking is a nicer framing than what I had in #6638 — it fixes #6635 without touching the interruption paths at all.

One thing worth pairing with it: once an uninterruptible speech is legitimately possible here, _on_input_speech_started calls self.interrupt() on every barge-in, and that call raises from the middle of AgentActivity.interrupt() — after _cancel_preemptive_generation() and _interrupt_background_speeches() have already run, but before the queued speeches, rt_session.interrupt() and the future wiring. The except RuntimeError here swallows the error, but the background speeches are already interrupted and the rest of the sequence is skipped, so each barge-in leaves a half-applied interruption. The same thing happens for a queued say(..., allow_interruptions=False), since the _speech_q loop doesn't filter protected handles the way _interrupt_background_speeches does.

Both are independent of which approach fixes #6635 (they reproduce today with a pipeline agent and say(allow_interruptions=False) + session.interrupt()), so I put them in #6644 rather than adding noise here.

[bml1g12]

turn_detection is given when using server_vad (create_response=False, interrupt_response=False,) VAD, but its still client driven turn taking, so I think can_disable_turn_detection should be True not False when create_response=False?

[longcw]

can_disable_turn_detection is true when turn_detection is not specified, it's not used if turn_detection is already False.

[bml1g12]

We have turn_detection = ServerVad() , i.e. specified, and have client controller turn detection (c.f. on #6635 (comment) "Note — why not LiveKit "manual" turn detection with no VAD" for why)

Which means I think can_disable_turn_detection = not isGiven(True) = False

can_disable_turn_detection: bool = False
"""Whether server-side turn detection can be disabled for a session so the client drives
turn-taking. Set by plugins that implement session(turn_detection_disabled=True)."""

Is can_disable_turn_detection = False problematic for addressing #6635 I wonder? I do not fully understand that flag, but I desire our own application code to control turn taking via RPC events, not the OpenAI server deciding when to generate replies and interrupt, so it sounds like I want it to be True to me?

[longcw]

can_disable_turn_detection means if the framework can automatically disable the server turn detection fully if a client turn detection is specified.

in your case, turn_detection = ServerVad() means we shouldn't change your config. and in this pr with create_response True, the cap.turn_detection is already set to False. nothing needed from can_disable_turn_detection

[bml1g12]

I see

[biztex]

@bml1g12 since you mentioned the flag is hard to place, here is your exact config traced through this PR — it should do what you want, and can_disable_turn_detection=False is the right value for you rather than a problem.

With turn_detection=ServerVad(create_response=False, interrupt_response=False):

1. capabilities.turn_detection becomes False, because this PR makes it resolved_turn_detection is not None and resolved_turn_detection.create_response is not False. (Small slip in the comment above — it's create_response False that flips the capability off, not True.)
2. AgentActivity._resolve_rt_turn_detection_enabled() returns on its first branch when caps.turn_detection is False, so can_disable_turn_detection is never read. That's what @longcw means by "nothing needed from it" — it only gates whether the framework may switch the server's turn detection off for you, and there is nothing left to switch off.
3. _rt_turn_detection_enabled is therefore False, so the two places that used to reject your setup no longer fire: the ValueError at construction (agent_activity.py:236) and the say() override that forced allow_interruptions back to NOT_GIVEN (agent_activity.py:1410). allow_interruptions=False is honoured, and say(..., allow_interruptions=False) keeps its value.

And because can_disable_turn_detection is False, the framework leaves the ServerVad config you passed exactly as you set it — with it True the framework would consider itself free to turn your server VAD off, which is the opposite of what you want, since you rely on the server VAD to keep committing and transcribing while your own code decides when to reply.

Worth keeping the interrupt_response=False in there as this PR's new warning suggests: with create_response=False alone the server still cancels its own response on user speech, which would cut a reply short even though your app never asked for it.

[bml1g12]

I see thanks for the detailed explanation, I misunderstood what the parameter can_disable_turn_detection meant