Skip to content

fix: guard choices access before indexing in LLM API calls (10 files) - #1057

Open
qizwiz wants to merge 2 commits into
meta-llama:mainfrom
qizwiz:fix/llm-response-unguarded-choices
Open

fix: guard choices access before indexing in LLM API calls (10 files)#1057
qizwiz wants to merge 2 commits into
meta-llama:mainfrom
qizwiz:fix/llm-response-unguarded-choices

Conversation

@qizwiz

@qizwiz qizwiz commented May 17, 2026

Copy link
Copy Markdown

Summary

LLM APIs (OpenAI, Groq, and OpenAI-compatible endpoints) can return an empty choices list in several legitimate scenarios:

  • Content filtering triggered (safety system returns no completions)
  • Rate-limit or quota errors during streaming
  • Network-level partial responses
  • Certain model configurations with n=0

Accessing response.choices[0] without a length check raises IndexError in these cases, turning a recoverable API error into an unhandled crash.

This PR adds an explicit if not <response>.choices guard before every unguarded choices[0] access across 10 files:

File Line Fix
3p-integrations/groq/.../groq-quickstart-conversational-chatbot/main.py 35 continue loop iteration
3p-integrations/groq/.../presidential-speeches-rag-with-pinecone/main.py 65 return ""
3p-integrations/groq/.../text-to-sql-json-mode/main.py 33 return ""
3p-integrations/groq/.../verified-sql-function-calling/main.py 135 return None
end-to-end-use-cases/Contextual-Chunking-RAG/helper.py 30, 40 return ""
end-to-end-use-cases/Multi-Modal-RAG/scripts/final_demo.py 115, 184 return fallback
end-to-end-use-cases/github_triage/llm.py 39 inline ternary guard
end-to-end-use-cases/research_paper_analyzer/research_analyzer.py 98 return []
end-to-end-use-cases/structured_parser/src/utils.py 456 raise ValueError
getting-started/inference/api_inference/api_inference.py 40 return ""

Found by pact static analysis (llm_response_unguarded mode), which uses a Z3 Fixedpoint Datalog engine to track unguarded LLM API response accesses across Python call graphs.

Test plan

  • Verify pact reports 0 llm_response_unguarded violations on patched files
  • Run existing example tests if any exist in CI

LLM APIs (OpenAI-compatible) can return empty choices lists on content
filtering, rate-limit errors, or streaming edge cases. Add an explicit
`if not <response>.choices` check before `choices[0]` access in 10 files
across groq templates, end-to-end examples, and getting-started scripts.

Found by pact static analysis (llm_response_unguarded mode).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@meta-cla

meta-cla Bot commented May 17, 2026

Copy link
Copy Markdown

Hi @qizwiz!

Thank you for your pull request and welcome to our community.

Action Required

In order to merge any pull request (code, docs, etc.), we require contributors to sign our Contributor License Agreement, and we don't seem to have one on file for you.

Process

In order for us to review and merge your suggested changes, please sign at https://code.facebook.com/cla. If you are contributing on behalf of someone else (eg your employer), the individual CLA may not be sufficient and your employer may need to sign the corporate CLA.

Once the CLA is signed, our tooling will perform checks and validations. Afterwards, the pull request will be tagged with CLA signed. The tagging process may take up to 1 hour after signing. Please give it that time before contacting us about it.

If you have received this in error or have any questions, please contact us at cla@meta.com. Thanks!

Gemini 2.5 Flash returns HTTP 200 with choices[0].message=None when its
safety filter triggers (finish_reason='content_filter: PROHIBITED_CONTENT').
The previous guard `if not response.choices` catches the empty-list case
but misses this: one choice is present but the message object is None,
causing AttributeError: 'NoneType' object has no attribute 'content'.

Reproduced live against Gemini 2.5 Flash — crash confirmed in unguarded
code, silent with the comprehensive guard.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@qizwiz

qizwiz commented May 17, 2026

Copy link
Copy Markdown
Author

Update: guard strengthened after live reproduction (commit 7e52db5)

While building a proof script, we found the original guard (if not response.choices) is incomplete for one additional failure mode.

Gemini 2.5 Flash (OpenAI-compatible endpoint) returns HTTP 200 with choices[0].message = None when its safety filter blocks a request — finish_reason: 'content_filter: PROHIBITED_CONTENT'. The choices list has one entry so the length guard passes, but accessing .message.content raises AttributeError: 'NoneType' object has no attribute 'content'.

# previous guard — still crashes on Gemini content filtering
if not response.choices:
    return ""
return response.choices[0].message.content  # AttributeError if message is None

# comprehensive fix (this commit)  
if not response.choices or response.choices[0].message is None:
    return ""
return response.choices[0].message.content  # safe

All 10 files updated. This covers: empty choices list (any provider), null message on content filter (Gemini and potentially others).

Note: still waiting on CLA confirmation before this can be reviewed — will sign at code.facebook.com/cla.

@qizwiz

qizwiz commented May 18, 2026

Copy link
Copy Markdown
Author

@meta-cla-bot recheck

@meta-cla

meta-cla Bot commented May 18, 2026

Copy link
Copy Markdown

Thank you for signing our Contributor License Agreement. We can now accept your code for this (and any) Meta Open Source project. Thanks!

@meta-cla meta-cla Bot added the cla signed label May 18, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant