Skip to content

adversarial-review / review can complete the turn without a schema-conforming final message on multi-tool-call reviews at high reasoning effort #496

Description

@yrevah1

Summary

adversarial-review (and review, same runAppServerTurn/turn/start path) can complete its Codex turn while never producing a schema-conforming final message. The turn genuinely finishes (turn/completed fires, exit status 0), but finalMessage is either an early interim agentMessage ("commentary" phase) or empty — so parseStructuredOutput fails every time with Unexpected token '...' is not valid JSON or Codex did not return a final structured message. This happened on 7 consecutive attempts against the same review target; only falling back to plain task (no outputSchema) with the same prompt succeeded.

Root cause (isolated via 3 controlled repros)

  1. No timeout margin for multi-tool-call reviews at high/xhigh reasoning effort. A review of a moderately sized diff (10 files, ~570 lines) needs several sequential rounds of git diff, per-file reads, and cross-referencing before Codex is ready to commit to a verdict. I ran the identical adversarial-review prompt unattended for 8 minutes (timeout 480) and Codex was still mid-review — reading the last diff hunk, about to move to source inspection — when the process was reaped, having spent ~42K tokens without ever reaching a final agentMessage. There is no visible per-job timeout or progress-based time extension in executeReviewRun/captureTurn, so the caller has no way to know the turn needs more time versus being stuck.

  2. A global ~/.codex/AGENTS.md skill-delegation directive derails/delays the turn further, but is not the root cause by itself. If the user's AGENTS.md contains an instruction like "use skill X for codebase-heavy work," Codex announces intent to delegate ("I'll use /X because this is codebase-heavy review work...") and stalls there for one or more turns — but no skill/subagent mechanism exists in the headless Codex CLI runtime, so this is pure wasted time. I confirmed this is a compounding factor, not the sole cause: re-running with a clean CODEX_HOME (no AGENTS.md) removed the skill-mention stall, but the review still ran out of time before completing on the same diff.

  3. The failure surfaces as a JSON parse error, not a timeout error. Because captureTurn's state.completion resolves once turn/completed notification fires (or via scheduleInferredCompletion), and the harness doesn't distinguish "turn completed with a real final answer" from "turn completed/was cut off with only interim commentary captured," the caller sees a confusing parseError instead of an actionable "the review needs more time" signal.

Repro

# From a repo with a branch carrying a non-trivial diff (~10 files):
node scripts/codex-companion.mjs adversarial-review --base <ref> --json "some focus text"

Result across repeated attempts: "parseError": "Unexpected token '...' is not valid JSON" or "rawOutput": "", despite "codex": {"status": 0} — the turn "succeeded" by the harness's own accounting but produced no usable structured output.

Isolating the timeout: running the same prompt directly via codex exec --output-schema <path> "<same adversarial-review prompt>" with an 8-minute wall clock and a clean CODEX_HOME (no AGENTS.md) still had not emitted a final answer when killed — confirming the turn's natural completion time for a moderate diff at high effort exceeds whatever margin the companion script currently assumes.

Suggested fix

  • Add an explicit, configurable timeout with a clear "review still running / needs more time" signal, distinct from a JSON-parse failure — e.g. detect finalMessage phase ≠ final_answer and surface status: "incomplete" instead of attempting to JSON.parse interim commentary.
  • For adversarial-review/review specifically, consider defaulting to a lower reasoning effort than the user's model_reasoning_effort (see review / adversarial-review silently ignore reasoning effort — --effort unparsed, and turn/start effort omitted on the adversarial path #476, which already tracks that effort isn't threaded through this path) or chunking large diffs, since exhaustive high-effort reasoning over a multi-file diff is what exhausts the available time budget.
  • Document that a user's ~/.codex/AGENTS.md skill/slash-command directives (meant for interactive sessions) can cost extra turns in headless companion runs where no such delegation mechanism exists; consider having the companion strip or override skill-invocation instructions for its own turns, or note this as a known interaction in the docs.

Environment

  • plugin codex@openai-codex v1.0.6
  • codex-cli 0.144.3
  • model openai.gpt-5.5 via amazon-bedrock provider, model_reasoning_effort = "high"
  • Linux, Claude Code session

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions