`gemini-3.1-flash-live-preview`: a turn left open after an ignored utterance never closes on its own; the caller's next sentence is consumed to close it with zero output tokens

[IngLP]

Description of the bug

On gemini-3.1-flash-live-preview with server-side (automatic) activity detection, the server
sometimes does not react to a caller utterance at all: no input transcription, no
interrupted, no generation, while the WebSocket is alive and session-resumption updates keep
arriving every ~0.9 s. That half is already reported by several people (links at the bottom).

The part we could not find reported anywhere is what the session does next:

1. The open turn never closes on its own. In 5 trials, after the ignored utterance the caller
   goes silent and we observe the socket for up to +13.3 s from the moment the utterance was
   swallowed. Nothing arrives in that window: no transcription, no interrupted, no
   turnComplete, no usageMetadata. Only sessionResumptionUpdate, every ~0.9 s.
2. What closes it is the caller speaking again. 5 measurements out of 5: the release arrives
   2.69–2.85 s after the start of the next utterance — which is the normal input
   transcription latency for a ~1.8 s phrase on this model — and the turn closes with zero
   output tokens (usageMetadata with no responseTokenCount).
3. Net effect: two caller utterances produce no answer. The first is ignored; the second is
   spent closing the stale turn and gets no generation of its own. If the caller keeps talking,
   the cycle can repeat.

We previously believed there was a fixed ~10.3 s "deaf window" after a swallowed utterance
(measured 10.29–10.35 s across 7 of 7 sessions, which looked very much like a server timer).
That was an artefact of our own probe's cadence: it declared the swallow at +6 s and started
speaking again at +7.8 s, and 7.8 + ~2.5 s of transcription latency ≈ 10.3 s. There is no timer.
Measured window lengths were 8.01 / 8.01 / 8.03 / 8.03 / 8.17 s — with the next utterance always
starting at exactly +5.32 s. The window tracks the caller's cadence, not the clock.

Related, and tested because the documentation suggests it should help: audioStreamEnd does
not flush anything. 0 reactions in 5 trials, 8 s of silent observation after each flush. No
transcription, no generation, no turn closed, no content recovered. The message is accepted
without error and the connection stays alive. 3 of those trials had the caller leg streaming
continuously (our production shape) and 2 had a full second of real silence around the flush, to
rule out the documented behaviour that sending audio reopens the stream and could cancel the
flush. Same result in both forms.

Actual vs expected behavior

	expected	observed
a turn opened on caller audio that the server does not act on	closes or errors within some bounded time	never closes while the caller is silent; observed to +13.3 s, 5/5
the caller's next utterance	opens its own turn and gets a generation	consumed to close the stale turn, 5/5, with no responseTokenCount
audioStreamEnd in automatic-VAD mode	flushes cached input audio, per the docs	no server reaction at all, 0/5

Environment

Model	gemini-3.1-flash-live-preview
Transport	google-genai Python SDK 1.73.1
Endpoint / backend	Gemini API — generativelanguage.googleapis.com, API key auth. Not Vertex AI.
Python / platform	CPython 3.12, macOS arm64
Audio	PCM s16le, 16 000 Hz, 1 channel, 16-bit; audio/pcm;rate=16000; 20 ms frames sent every 20 ms, continuously, digital zeros during pauses — the sender never stops
Activity detection	server default in the campaigns that measure the production configuration; a pinned automaticActivityDetection in the older rate campaign, stated per number below
realtimeInputConfig	activityHandling: START_OF_ACTIVITY_INTERRUPTS, turnCoverage: TURN_INCLUDES_ONLY_ACTIVITY (also tested with TURN_INCLUDES_ALL_INPUT, no difference — see below)
Also configured	inputAudioTranscription: {}, outputAudioTranscription: {}, sessionResumption, systemInstruction ~60 000 chars

Exact UTC timestamps of the sessions cited here are available on request, and we are happy to
provide our Google Cloud project number privately through whichever intake form you prefer — it
is deliberately not in this public body.

Evidence

Probe timeline of one control trial, verbatim (t in seconds from session start; ### lines are
the probe's own markers, the others are received LiveServerMessages):

   9.27 ### speak #1 dt_gc=5.92
  14.59 ### swallowed start=9.27 detected=14.59
  14.59 ### retry #1 (+5.32s)
  17.10 ### retry #2 (+7.83s)
  17.44 in_tx 'Sì, ci sono.'
  17.82 interrupted
  17.83 turn_complete
  17.83 usage prompt=15695 out=None

The utterance at 9.27 s is ignored (nothing between 9.27 and 14.59). The transcription at 17.44 s
arrives 2.85 s after the start of retry #1 (14.59) and only 0.34 s after the start of
retry #2 (17.10) — far too early to be retry #2, so it is the release triggered by
retry #1. The turn then closes with out=None.

The audioStreamEnd arm of the same campaign, for contrast: nothing at all is received between
the flush at 14.33 s and the end of observation at 21.82 s except session-resumption updates.

How often the state is entered — and why there is no deterministic recipe

We do not have a deterministic reproducer for the ignored utterance itself. It is stochastic;
what we have are measured rates, and they move a lot. Stated with the configuration each was
measured under:

population	configuration	rate
~1.8 s utterances, 100 extractions	pinned automaticActivityDetection (prefixPaddingMs=600, silenceDurationMs=500, start HIGH, end LOW)	29 %
~0.6 s utterances, 100 extractions	same pinned configuration	9 %
utterances landing 5.0–6.8 s after the previous generationComplete, 12 observations	no automaticActivityDetection block at all (server default)	42 % (5/12)
same risk band, 29 observations, later campaign	server default	24 % (7/29)

Caveat we would repeat to anyone quoting these numbers: the rate oscillates strongly at fixed
configuration. Two consecutive sessions of one campaign gave 0 swallows out of 8, and another
pair shortly afterwards gave 3 out of 10, same config, same audio, same minutes. Treat these
as orders of magnitude, not point estimates.

Recipe that gets you there stochastically, with the config above:

1. Stream 16 kHz PCM continuously, 20 ms frames, zeros during pauses, never stopping.
2. Have the caller speak ~1.8 s utterances, each starting about 5.0–6.8 s after the previous
   generationComplete. That band is where we see the ignored utterances concentrate.
3. Roughly one utterance in three or four gets no reaction at all. When it does, stay silent
   and watch the socket: that is the state this report is about.
4. Then speak again and watch the transcription of the new utterance close the old turn with
   out=None.

Two negative results from the same work, so nobody re-runs them:

• turnCoverage: TURN_INCLUDES_ALL_INPUT changes neither the rate (3/10, 30 %, against 7/26,
  27 %, measured in the same minutes) nor the window (8.01 / 8.03 / 8.01 s against 8.03 / 8.17 s).
• startOfSpeechSensitivity=HIGH (4/13, 31 %) and prefixPaddingMs=150 (4/14, 29 %) do not
  separate from the 42 % baseline at this sample size; inside the prefixPaddingMs arm alone the
  rate goes from 0/5 in one round to 2/3 in another at identical parameters.

The audio stream is continuous — this is measured, not asserted

This is the first objection raised on similar reports (e.g.
dotnet-genai#269, closed after the
reporter's stream turned out to have gaps), so:

• Every probe here uses the same caller leg: one 20 ms PCM frame every 20 ms for the whole
  session, digital zeros during pauses, never stopping — deliberately modelled on a telephone
  line, which never goes silent on the wire.
• The standalone reproducer of our other report (cookbook#1328, script and output in https://gist.github.com/IngLP/124873f0574ada091ab4324e5ff12432) instruments that
  sender and prints the numbers. Over a 63.3 s session: 3150 chunks sent, ~3166 expected, mean
  inter-chunk interval 20.0 ms, worst interval 22.2 ms, against a 20 ms nominal period. There is
  no gap anywhere for the server to read as end of input.
• Independent cross-check on the same client, same audio path, same network: with
  automaticActivityDetection.disabled = true and explicit activityStart / activityEnd
  markers driven by a local VAD, the swallow rate in our lab drops from 29 % / 9 % to 1 % / 0 %,
  and 115 turns out of 115 were answered across three long sessions. Only the segmentation
  authority changed. (We are not running that mode in production because it costs us ~2.3–2.4 s
  of perceived latency against ~1.74 s in automatic mode — mentioning it only as evidence about
  where the fault sits.)

Related issues

The ignored-utterance half, reported in pieces:

• cookbook#1197 "Issue 9" — 567 audio chunks received, 0 inputTranscription events on an 11 s call
• live-api-web-console#142 — short utterances never committed by the VAD; extended to 3.1 in a 2026-04-25 comment
• python-genai#2580 — silence_duration_ms ignored on 3.1
• forum 146852 — VAD deaf for the first 10–17 s of a session
• forum 144747 — audio in, only sessionResumptionUpdate out
• python-genai#2333 (closed by the stale bot) — phantom interrupted=True wedging the session into a zero-audio state

We could not find the never-closing turn, or the next utterance being consumed to close it empty,
in any of these.

We are filing two other reports on the same model. They may share a root cause with this one, but
we have not demonstrated that: cookbook#1328, and a comment with a voice_activity repro on
python-genai#1285. The missing
voice_activity signal is what makes this particular state undiagnosable from the client: there
is no server-side event that says whether the turn is still open.

Questions

1. Is a turn opened on caller audio expected to remain open indefinitely when the server produces
   no transcription and no generation for it?
2. Is audioStreamEnd supposed to have an effect in automatic-VAD mode? If it is a no-op there
   by design, the documentation currently reads as though it flushes cached input audio.
3. Is there any client-observable signal that distinguishes "the server is still processing this
   utterance" from "this utterance will never be answered"? Today we detect it only by absence,
   with a timeout.

[kkorpal]

Hey @IngLP, thanks for sharing such a thorough breakdown. I'm taking this directly to the engineering team for them to investigate. Really appreciate you flagging this.