google_search grounding silently disabled when structured/JSON output is requested (gemini-3.5-flash)

[sdtnjung]

Description

google_search grounding is silently disabled whenever structured / JSON output is requested on the same generateContent call. The request succeeds (HTTP 200) and returns a well-formed answer, but candidates[0].groundingMetadata is absent — no webSearchQueries, no groundingChunks — so the model answers from parametric memory only. With an identical prompt + tool but plain-prose output, grounding fires reliably.

This is easy to miss because there is no error and no warning; the only signal is the missing groundingMetadata. For any pipeline that needs both fresh web facts and a structured result, this forces an awkward two-pass workaround (ground in prose, then re-prompt to structure into JSON).

Model

models/gemini-3.5-flash

Steps to reproduce

Two requests, identical except for the output mode. Both attach the google_search tool and ask a question whose answer cannot be known without searching (a live stock price move), so grounding is genuinely required.

A) Prose output — grounding FIRES ✅

curl -s "https://generativelanguage.googleapis.com/v1beta/models/gemini-3.5-flash:generateContent?key=$GEMINI_API_KEY" \
  -H 'Content-Type: application/json' \
  -d '{
    "contents": [{"role":"user","parts":[{"text":"How much did REGENXBIO (RGNX) stock move in pre-market trading today, and why? Answer in one short sentence."}]}],
    "tools": [{"google_search": {}}],
    "generationConfig": {"temperature": 0.1}
  }' | jq '{grounded: (.candidates[0].groundingMetadata != null), queries: .candidates[0].groundingMetadata.webSearchQueries}'
# => { "grounded": true, "queries": ["RGNX premarket ...", ...] }

B) JSON output — grounding SILENTLY DISABLED ❌

curl -s "https://generativelanguage.googleapis.com/v1beta/models/gemini-3.5-flash:generateContent?key=$GEMINI_API_KEY" \
  -H 'Content-Type: application/json' \
  -d '{
    "contents": [{"role":"user","parts":[{"text":"How much did REGENXBIO (RGNX) stock move in pre-market trading today, and why? Respond as JSON {\"move_pct\": number, \"reason\": string}."}]}],
    "tools": [{"google_search": {}}],
    "generationConfig": {
      "temperature": 0.1,
      "responseMimeType": "application/json",
      "responseSchema": {"type":"object","properties":{"move_pct":{"type":"number"},"reason":{"type":"string"}}}
    }
  }' | jq '{grounded: (.candidates[0].groundingMetadata != null), queries: .candidates[0].groundingMetadata.webSearchQueries}'
# => { "grounded": false, "queries": null }

Expected behavior

google_search grounding should fire independently of the output format. A request that attaches the tool and asks an unknowable question should ground in both prose and JSON modes.

Actual behavior

Grounding fires in prose mode and is silently suppressed as soon as structured output is requested. We reproduced the suppression with all three structured-output mechanisms:

• responseMimeType: "application/json" + responseSchema
• responseJsonSchema
• JSON merely requested via the prompt text (no responseMimeType)

It is deterministic across repeated calls: in our test harness, JSON output grounded 0/5 times and prose grounded 5/5 times with otherwise identical prompts. Even adding an explicit "you MUST use Google Search" instruction did not make JSON-mode requests ground.

Impact / workaround

Because there is no error, callers can ship code that appears grounded but answers from stale parametric memory — in our case a news-trading bot judged an already-priced-in catalyst as fresh. Current workaround: a two-pass flow — pass 1 grounds in prose, pass 2 re-prompts (no tool) to structure that prose into JSON. Doubles latency and token cost.

Questions

1. Is this an intended limitation (structured output and google_search are mutually exclusive)?
2. If so, can the API return an explicit error / warning instead of silently dropping the tool, so callers aren't misled?
3. Is a single-call grounded-JSON path on the roadmap?

[kkorpal]

Hi @sdtnjung, Reproduced locally. When responseMimeType: "application/json" is enabled on gemini-3.5-flash, the API silently drops the groundingMetadata block entirely and falls back to training memory without throwing an error or warning. Thanks for bringing this to us! I will take this to the engineering team.