Bishop - CLI Spec v1

CLI tool that verifies and addresses PR review comments from code review bots by dispatching each comment to a coding agent for verification and fixing.

Overview

Bishop auto-detects the PR from the currently checked-out branch, fetches inline review comments left by configured bots, and sequentially invokes a coding agent to verify each issue and fix it if real. It can also fix configured CI check failures by passing workflow context to the agent.

Designed to work both locally and in GitHub Actions (single-iteration model; in CI, re-triggered by push events rather than long-running loops).

CLI Interface

`bishop`

Main command. Processes all unresolved, non-outdated inline review comments from configured bots, then fixes configured CI check failures.

bishop [flags]

Flags:
  --no-commit    Skip committing fixes (overrides config)
  --no-push      Skip pushing to remote (overrides config, implies no CI check)
  --no-reply     Skip posting replies on GitHub comments (overrides config)
  --dry-run      Show what comments/checks would be processed without invoking the agent

`bishop init`

Generates a default bishop.json config file in the repo root.

Configuration

File: bishop.json at repo root. Validated at startup with Zod; invalid config produces clear error messages and exits.

{
	"bots": ["coderabbitai"],
	"checks": ["CI / build", "CI / lint"],
	"agent": "cursor",
	"commit": true,
	"push": true,
	"reply": true
}

Field	Type	Default	Description
`bots`	`string[]`	`[]`	GitHub usernames of review bots whose comments bishop should process
`checks`	`string[]`	`[]`	Check/job names (as shown by `gh pr checks`) to monitor and fix on failure
`agent`	`string`	`"cursor"`	Coding agent adapter to use
`commit`	`boolean`	`true`	Whether to commit each fix. Overridden by `--no-commit`
`push`	`boolean`	`true`	Whether to push after all fixes. Overridden by `--no-push`
`reply`	`boolean`	`true`	Whether to post replies on GitHub comments. Overridden by `--no-reply`

Execution Flow

Startup

Load and validate bishop.json from repo root (Zod schema validation)
Verify gh CLI is authenticated (gh auth status)
Verify git working tree is clean (no uncommitted changes); abort if dirty
Auto-detect PR from current branch via gh pr view --json number; error and exit if no PR found

Phase 1: Bot Comments

Fetch all inline review comments on the PR via GitHub API
Filter to: comments from configured bots, not resolved, not outdated
Report summary: how many comments found from each bot
If --dry-run: print the list of comments that would be processed and exit

For each comment (sequentially):

Build rich context for the agent:
- Comment body text
- File path and line number
- Diff hunk / surrounding code context
- Full thread (all replies in the conversation)
Invoke the coding agent with a prompt instructing it to verify the issue and fix if real
Parse the agent's structured JSON response from stdout to determine:
- issueExists: whether the issue is real
- summary: description of what was done or why it's not an issue
If issue exists and was fixed:
- If commit enabled: git add -A && git commit -m "bishop: {summary}"
- Post reply on GitHub: details of what was fixed, referencing the commit
If issue does not exist (false positive):
- Post reply on GitHub: explanation of why the issue is not real
If agent fails (crash, timeout, invalid output): log failure, skip to next comment

Phase 2: CI Checks

If push is enabled and there are fixes to push: git push
For each configured check name in checks:
- Query check status via gh pr checks
- If the check failed:
  - Build prompt with the workflow/check name and instruct the agent to use gh CLI to fetch logs and investigate
  - Invoke coding agent to diagnose and fix
  - If fixed and commit enabled: commit the fix
If any CI fixes were committed and push enabled: push again

Completion

Print colored summary table:
- Per-comment: file:line, bot name, status (fixed/false-positive/skipped/failed)
- Per-check: check name, status (passed/fixed/failed)
Exit with code 0

Agent Adapter System

Pluggable interface for coding agent integrations. Start with Cursor CLI adapter.

Interface

interface AgentResult {
	issueExists: boolean;
	summary: string;
}

interface AgentAdapter {
	name: string;
	invoke(prompt: string, options: AgentInvokeOptions): Promise<AgentResult>;
}

interface AgentInvokeOptions {
	workingDirectory: string; // always repo root
}

Cursor CLI Adapter

Invokes Cursor CLI in headless print mode:

agent -p --force --output-format json "<prompt>"

The prompt instructs the agent to:

Verify whether the reported issue actually exists in the code
If it exists, fix it
Output a JSON block at the end of its response with the structure:

[BISHOP_RESULT]
{"issueExists": true, "summary": "Fixed null check in auth handler"}
[/BISHOP_RESULT]

Bishop parses the agent's JSON output, extracts the result text field, and looks for the [BISHOP_RESULT]...[/BISHOP_RESULT] markers to extract the structured result.

Adding New Adapters

New agents can be added by implementing the AgentAdapter interface. The adapter handles CLI invocation, output parsing, and result extraction. The agent field in bishop.json selects which adapter to use.

Agent Prompt Design

Comment Verification Prompt

Hardcoded in source (not user-configurable). Structure:

You are a code review assistant. A review bot has flagged an issue in a pull request.

## Review Comment
Bot: {bot_name}
File: {file_path}:{line_number}
Comment: {comment_body}

## Thread Context
{thread_replies}

## Diff Context
{diff_hunk}

## Instructions
1. Read the relevant source code and verify whether this issue actually exists
2. If the issue exists, fix it
3. If the issue does not exist (false positive), explain why

## Output
After completing your analysis, output your result in this exact format:

[BISHOP_RESULT]
{"issueExists": <true|false>, "summary": "<what you did or why it's not an issue>"}
[/BISHOP_RESULT]

CI Fix Prompt

A CI check has failed on this pull request.

Check name: {check_name}
Workflow: {workflow_name}

Use the `gh` CLI to fetch the workflow run logs and investigate the failure.
You can use commands like:
- `gh run view <run_id> --log-failed`
- `gh run view <run_id> --log`

Diagnose the failure and fix the code.

[BISHOP_RESULT]
{"issueExists": <true|false>, "summary": "<what you fixed or why it failed>"}
[/BISHOP_RESULT]

GitHub API Usage

All GitHub interaction goes through the gh CLI (leveraging its existing authentication).

Key commands:

gh pr view --json number,headRefName - detect PR from branch
gh api repos/{owner}/{repo}/pulls/{pr}/comments - fetch inline review comments
gh pr checks - get check statuses
gh api repos/{owner}/{repo}/pulls/{pr}/comments -f body=... - post reply to review comment
gh run view <id> --log-failed - get failed CI logs (used by agent directly)

Comment Processing Rules

Only inline review comments (comments on specific diff lines) are processed
Skip resolved comments - already handled
Skip outdated comments - code has changed since comment was posted
Process all bots independently - even if multiple bots comment on the same line, each is processed separately; the agent will recognize if a prior fix already addressed the issue
Sequential processing - one comment at a time to avoid file conflicts

Error Handling

Scenario	Behavior
`bishop.json` missing/invalid	Print validation error, exit
`gh` CLI not authenticated	Print error with `gh auth login` hint, exit
Dirty git working tree	Print error listing dirty files, exit
No PR for current branch	Print error, exit
No comments from configured bots	Print "no comments found", exit normally
Agent fails on a comment	Log failure, skip to next comment
Agent returns unparseable output	Treat as agent failure, skip
`git commit` fails	Log error, skip to next comment
`git push` fails	Log error, print warning
GitHub API rate limit	Log error, exit

Output Format

Colored terminal output during execution:

bishop v0.1.0

Config loaded: 2 bots, 2 checks, agent: cursor
PR #42: feature/add-auth (on branch feature/add-auth)
Working tree: clean

Found 5 comments from configured bots:
  coderabbitai: 4 comments
  codeclimate: 1 comment

[1/5] coderabbitai - src/auth.ts:42
      Verifying issue...
      Issue exists. Fixing...
      Fixed. Committed: bishop: add null check for token validation
      Replied on GitHub.

[2/5] coderabbitai - src/auth.ts:87
      Verifying issue...
      Not an issue (false positive).
      Replied on GitHub.

[3/5] coderabbitai - src/db.ts:15
      Verifying issue...
      Agent failed. Skipping.

...

Checking CI: CI / build... passed
Checking CI: CI / lint... failed
      Fixing CI failure...
      Fixed. Committed: bishop: fix lint errors in auth module

Pushing to origin...
Pushed.

Summary:
  Comments: 3 fixed, 1 false positive, 1 failed
  Checks:   1 passed, 1 fixed

Tech Stack

Runtime: Bun
Language: TypeScript
Config validation: Zod
GitHub interaction: gh CLI (shelled out via Bun.$)
Git operations: git CLI (shelled out via Bun.$)
Distribution: npm package

Project Structure

bishop/
  src/
    index.ts              # CLI entry point, argument parsing
    config.ts             # Config loading and Zod schema
    github.ts             # GitHub API interactions via gh CLI
    git.ts                # Git operations (status, commit, push)
    comments.ts           # Comment fetching, filtering, context building
    checks.ts             # CI check status and fix orchestration
    engine.ts             # Main orchestration loop
    prompt.ts             # Agent prompt templates
    adapters/
      types.ts            # AgentAdapter interface, AgentResult type
      cursor.ts           # Cursor CLI adapter
    output.ts             # Colored terminal output formatting
  bishop.json             # Example config (for reference)
  package.json
  tsconfig.json

FilesExpand file tree

SPEC.md

Latest commit

History