Context
Branch: hybrid-agent-cli
Wave: 1 (critical, blocks protocol integrity)
Found by: polytest run 20260423-133619, hit on 5 repos
Bug
The contribute workflow agent bypasses the polyresearch CLI commands and uses gh directly to create PRs, push branches, and interact with GitHub. It skips polyresearch attempt entirely, meaning no experiment result is recorded in the protocol trail. It skips polyresearch submit, creating PRs manually with gh pr create.
This breaks the entire protocol chain: no attempt comment on the issue, no metric in the ledger, the lead cannot decide the PR (no attempt to evaluate), and results.tsv is never updated.
Repos affected
- DOMPurify — agent created PR without
polyresearch submit, bypassed attempt
- litellm — agent bypassed both
polyresearch attempt and polyresearch submit, created PR with gh pr create
- remarkable — agent skipped
polyresearch attempt, no experiment data recorded
- markdown-it — no attempt recorded, results.tsv empty
- tokenizers — released thesis without recording attempt
Evidence (litellm)
The lead could not decide PR #6 because no attempt was recorded:
Lead cannot decide PR #6: no improved attempt recorded for the thesis branch.
The PR was created manually by the agent with gh pr create instead of polyresearch submit.
Evidence (DOMPurify)
PR created and merged, but results.tsv shows "improved" for an attempt that was later policy-rejected. The attempt was recorded inconsistently because the agent mixed CLI and direct gh calls.
Evidence (remarkable)
Contribute ran for full cycle, released thesis, but no polyresearch attempt comment exists on the issue. The experiment ran successfully but the result was never recorded in the protocol.
Root cause
The contribute workflow prompt (contribute-workflow.md) instructs the agent to use CLI commands, but the agent has full shell access and sometimes decides to use gh directly — either because it's more familiar with gh, or because it encounters an error with the CLI command and falls back to gh.
Possible fixes
- The experiment prompt should be more forceful about requiring CLI commands
- The
polyresearch submit command could be a hard gate: refuse to create a PR unless a valid attempt exists
- Consider restricting the agent's tool access to only allow
polyresearch commands (not raw gh)
Context
Branch:
hybrid-agent-cliWave: 1 (critical, blocks protocol integrity)
Found by: polytest run
20260423-133619, hit on 5 reposBug
The contribute workflow agent bypasses the
polyresearchCLI commands and usesghdirectly to create PRs, push branches, and interact with GitHub. It skipspolyresearch attemptentirely, meaning no experiment result is recorded in the protocol trail. It skipspolyresearch submit, creating PRs manually withgh pr create.This breaks the entire protocol chain: no attempt comment on the issue, no metric in the ledger, the lead cannot decide the PR (no attempt to evaluate), and results.tsv is never updated.
Repos affected
polyresearch submit, bypassed attemptpolyresearch attemptandpolyresearch submit, created PR withgh pr createpolyresearch attempt, no experiment data recordedEvidence (litellm)
The lead could not decide PR #6 because no attempt was recorded:
The PR was created manually by the agent with
gh pr createinstead ofpolyresearch submit.Evidence (DOMPurify)
PR created and merged, but results.tsv shows "improved" for an attempt that was later policy-rejected. The attempt was recorded inconsistently because the agent mixed CLI and direct
ghcalls.Evidence (remarkable)
Contribute ran for full cycle, released thesis, but no
polyresearch attemptcomment exists on the issue. The experiment ran successfully but the result was never recorded in the protocol.Root cause
The contribute workflow prompt (
contribute-workflow.md) instructs the agent to use CLI commands, but the agent has full shell access and sometimes decides to useghdirectly — either because it's more familiar withgh, or because it encounters an error with the CLI command and falls back togh.Possible fixes
polyresearch submitcommand could be a hard gate: refuse to create a PR unless a valid attempt existspolyresearchcommands (not rawgh)