Add GitHub issue triage skill - #1580
Conversation
dosubot
Bot
added
size:L
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
WalkthroughAdds the 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 5f2b53de0a
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
Pull request overview
Adds a repository-specific repo-issue-triage skill to standardize how maintainers classify, reproduce, diagnose, verify intent, and (optionally) fix GitHub issue reports in the Valibot monorepo, with explicit safeguards around untrusted input and outward-facing writes.
Changes:
- Introduces the
repo-issue-triageskill entrypoint (SKILL.md) with guardrails, an evidence-gated workflow, and recommendation guidance. - Adds phase-specific playbooks for reproduction, diagnosis, verification, and post-triage fixing (
reproduce.md,diagnose.md,verify.md,fix.md).
Reviewed changes
Copilot reviewed 5 out of 5 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| .agents/skills/repo-issue-triage/SKILL.md | Defines the triage workflow, guardrails, report structure, and recommendation process. |
| .agents/skills/repo-issue-triage/reproduce.md | Reproduction-only guidance, including harness selection and outcome classification. |
| .agents/skills/repo-issue-triage/diagnose.md | Diagnosis-only guidance for tracing causal runtime/type paths and assigning confidence. |
| .agents/skills/repo-issue-triage/verify.md | Intent/contract verification guidance and verdict taxonomy. |
| .agents/skills/repo-issue-triage/fix.md | Opt-in fixing guidance after triage, including test/validation expectations and boundaries. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 11
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In @.agents/skills/repo-issue-triage/fix.md:
- Line 53: Update the outward-action restriction in the issue-triage
instructions to include edit, assign, and reopen alongside the existing
prohibited actions. Reference the complete shared guardrail, or explicitly state
that no GitHub write for each action is allowed without separate user approval.
- Around line 9-10: Update the issue-triage requirements in the documented
workflow to require completed reproduction, diagnosis, and contract-verification
sections before any source edits. Proceed only when verification explicitly
identifies a contract violation or documentation defect; stop for not-a-bug,
unclear, or low-confidence results, leaving no speculative edits or knowingly
failing tests.
- Around line 25-31: Update the regression-test workflow to branch for
documentation bug reports and documentation fixes. For documentation changes,
require an appropriate docs build, link check, or API-documentation validation
instead of always requiring an owning-module regression test. Require
before/after regression-test evidence only when an applicable test exists, while
preserving the existing runtime, type, and distribution test guidance for code
fixes.
- Around line 35-40: Update the formatting workflow instructions around the
ESLint and Prettier commands to be baseline-safe: only run in-place fixes on
files clean at the baseline, or perform fixes in an isolated copy and apply only
the approved diff. Preserve pre-existing worktree changes and ensure final
status comparison can detect that no unrelated hunks were rewritten.
In @.agents/skills/repo-issue-triage/reproduce.md:
- Around line 8-10: Update the Intake and reproduction instructions to inspect
git status for relevant dirty changes before treating the current checkout as a
baseline. Reproduce from an isolated copy of the recorded HEAD when tracked
files, tests, configuration, lockfiles, or build outputs are modified; otherwise
explicitly mark the result as limited rather than claiming it reflects the
recorded commit.
- Around line 36-47: Update the reproduction guidance around the Vitest,
package-manager, build, and runtime commands to require non-interactive
execution inside a disposable, resource-limited environment with explicit
timeouts, resource limits, network restrictions, and no credential access. If
those isolation requirements cannot be provided, instruct triage to return
blocked-by-environment rather than running the reproduction unsafely.
In @.agents/skills/repo-issue-triage/SKILL.md:
- Around line 17-18: Update the triage instructions for creating report.md and
comment.md to detect whether the target directory or files already exist before
writing. Preserve any pre-existing ignored files by explicitly resuming the
existing run or selecting an exclusive new run directory, and never overwrite
user or prior-run evidence.
- Around line 13-14: Use a single dependency-installation policy across the
triage guidance: in .agents/skills/repo-issue-triage/SKILL.md lines 13-14,
require lifecycle scripts to remain disabled by default and allow exceptions
only in a disposable sandbox with no credentials, restricted network access, and
resource limits; in .agents/skills/repo-issue-triage/reproduce.md lines 11-12,
extend the same restriction from external or published reproductions to
current-checkout installs.
- Around line 23-32: Define and validate one canonical, repository-scoped run
identifier before creating triage artifacts: resolve the GitHub issue number
from numeric or URL intake, and generate a sanitized collision-resistant
identifier for offline investigations. In
.agents/skills/repo-issue-triage/SKILL.md lines 23-32, use that identifier for
the triage directory and create report.md exclusively; in
.agents/skills/repo-issue-triage/reproduce.md lines 29-34, reuse the same
identifier for temporary test names. Ensure all paths reject traversal and
collisions.
- Line 111: Update the approval-gated write procedure around the triage comment
draft to refetch and compare the complete issue snapshot immediately before each
approved write, including the title, updatedAt, body, comments, labels, state,
and every field used by the recommendation. Stop and request re-review whenever
any material field differs from the intake snapshot.
- Line 12: Update the repository-selection flow in repo-issue-triage so it
resolves owner/name once, whether from the current checkout or a user-specified
repository, then pass that resolved repository explicitly via --repo to every gh
read command and approved write command, including the commands associated with
the referenced examples. Keep the existing command behavior unchanged aside from
making repository targeting explicit.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro Plus
Run ID: 969d0a01-e5da-47eb-bab2-14e918fdd39c
📒 Files selected for processing (5)
.agents/skills/repo-issue-triage/SKILL.md.agents/skills/repo-issue-triage/diagnose.md.agents/skills/repo-issue-triage/fix.md.agents/skills/repo-issue-triage/reproduce.md.agents/skills/repo-issue-triage/verify.md
- Route non-reproduced outcomes and unmet phase gates directly to the recommendation with skipped phases recorded - Define an offline artifact scheme and skip GitHub-only steps for investigations without an issue number - Make documentation updates part of public API fixes by default - Use explicit pnpm package context in reproduction commands - Use stable verdict tokens in the fix boundary
![cubic-dev-ai[bot]](https://avatars.githubusercontent.com/in/1082092?s=60&v=4){kind=small}
There was a problem hiding this comment.
All reported issues were addressed
Reply with feedback, questions, or to request a fix.
Re-trigger cubic
- Skip the regression-test requirement for documentation-only fixes and validate with build and lint checks instead - Reference the complete outward-action guardrail in the fix workflow - Isolate or flag reproductions when pre-existing modifications could influence the result - Run reproductions non-interactively with bounded timeouts - Pass --repo explicitly when targeting another repository - Resume existing triage directories instead of overwriting findings and sanitize offline slugs - Include the title and all recommendation-relevant fields in the pre-write revalidation
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: f95a809135
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
All reported issues were addressed across 3 files (changes from recent commits).
Tip: Review your code locally with the cubic CLI to iterate faster.
Re-trigger cubic
- Allow documentation-only fixes for standalone documentation reports gated by a confirmed source/documentation mismatch - Recommend and support closing verified duplicates as duplicate - Substitute the offline slug in temporary test filenames and commands - Correct the utils/ landmark to include public utilities
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: b1225e837a
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
All reported issues were addressed across 3 files (changes from recent commits).
Tip: Review your code locally with the cubic CLI to iterate faster.
Re-trigger cubic
- Namespace triage directories by repository when targeting a repository other than the current checkout and confirm the recorded issue URL before resuming - Add a reopen recommendation and command for re-triaged issues
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: e68b43f534
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
- Limit named repositories to the same project, such as an upstream or fork, and record checkout divergence as a limitation - Substitute the repository-scoped triage directory everywhere the default path appears - Make the reproduction outcome conditional in the triage summary for reports that skip reproduction
There was a problem hiding this comment.
1 issue found across 1 file (changes from recent commits).
Prompt for AI agents (unresolved issues)
Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.
<file name=".agents/skills/repo-issue-triage/SKILL.md">
<violation number="1" location=".agents/skills/repo-issue-triage/SKILL.md:32">
P2: The new `<owner>-<repo>-<number>` directory naming convention for triaging a non-current-checkout repository is only introduced here, but the phase files (reproduce.md, diagnose.md, verify.md, fix.md) and the draft/comment write instructions later in this file still reference `tmp/triage/gh-<number>/` directly. This inconsistency can cause an agent to split one investigation across two different directories or post from the wrong draft when `--repo` targets another repository. Consider defining the same explicit substitution rule everywhere the `gh-<number>` placeholder path is used, similar to how offline slugs are handled.</violation>
</file>
Tip: Review your code locally with the cubic CLI to iterate faster.
Re-trigger cubic
| gh label list --limit 100 --json name,description | ||
| ``` | ||
|
|
||
| Create `tmp/triage/gh-<number>/report.md`. When triaging a repository other than the current checkout, include the repository in the directory name, such as `tmp/triage/<owner>-<repo>-<number>/`. If the triage directory already exists from an earlier run, confirm its recorded issue URL matches, then resume it and append instead of overwriting earlier findings. For an offline investigation without an issue number, derive a short sanitized slug from the topic, use `tmp/triage/offline-<slug>/` everywhere `tmp/triage/gh-<number>/` appears, substitute the slug for `<number>` in temporary test filenames and commands, skip live label lookups and other GitHub-only steps, and deliver the recommendation — including any draft comment — to the user instead of preparing GitHub writes. Keep these sections in order and append evidence without rewriting earlier phase findings: |
There was a problem hiding this comment.
P2: The new <owner>-<repo>-<number> directory naming convention for triaging a non-current-checkout repository is only introduced here, but the phase files (reproduce.md, diagnose.md, verify.md, fix.md) and the draft/comment write instructions later in this file still reference tmp/triage/gh-<number>/ directly. This inconsistency can cause an agent to split one investigation across two different directories or post from the wrong draft when --repo targets another repository. Consider defining the same explicit substitution rule everywhere the gh-<number> placeholder path is used, similar to how offline slugs are handled.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At .agents/skills/repo-issue-triage/SKILL.md, line 32:
<comment>The new `<owner>-<repo>-<number>` directory naming convention for triaging a non-current-checkout repository is only introduced here, but the phase files (reproduce.md, diagnose.md, verify.md, fix.md) and the draft/comment write instructions later in this file still reference `tmp/triage/gh-<number>/` directly. This inconsistency can cause an agent to split one investigation across two different directories or post from the wrong draft when `--repo` targets another repository. Consider defining the same explicit substitution rule everywhere the `gh-<number>` placeholder path is used, similar to how offline slugs are handled.</comment>
<file context>
@@ -29,7 +29,7 @@ gh issue view <number> --json number,title,body,state,url,author,labels,comments
-Create tmp/triage/gh-<number>/report.md. If the triage directory already exists from an earlier run, resume it and append instead of overwriting earlier findings. For an offline investigation without an issue number, derive a short sanitized slug from the topic, use tmp/triage/offline-<slug>/ everywhere tmp/triage/gh-<number>/ appears, substitute the slug for <number> in temporary test filenames and commands, skip live label lookups and other GitHub-only steps, and deliver the recommendation — including any draft comment — to the user instead of preparing GitHub writes. Keep these sections in order and append evidence without rewriting earlier phase findings:
+Create tmp/triage/gh-<number>/report.md. When triaging a repository other than the current checkout, include the repository in the directory name, such as tmp/triage/<owner>-<repo>-<number>/. If the triage directory already exists from an earlier run, confirm its recorded issue URL matches, then resume it and append instead of overwriting earlier findings. For an offline investigation without an issue number, derive a short sanitized slug from the topic, use tmp/triage/offline-<slug>/ everywhere tmp/triage/gh-<number>/ appears, substitute the slug for <number> in temporary test filenames and commands, skip live label lookups and other GitHub-only steps, and deliver the recommendation — including any draft comment — to the user instead of preparing GitHub writes. Keep these sections in order and append evidence without rewriting earlier phase findings:
- Intake — issue URL, last reviewed update, category, affected package, current labels, observed behavior, expected behavior, environment, and missing facts.
</file context>
</details>
commit: |
Summary
Why
This gives maintainers a reusable workflow for classifying reports, reproducing runtime and TypeScript bugs, finding root causes, distinguishing defects from intended behavior, and drafting issue responses without forcing every report toward a fix.
The workflow is informed by Cloudflare's Astro issue-triage write-up and Astro's open-source triagebot-action, adapted for interactive, human-approved maintenance of Valibot.
Validation
pnpm exec prettier --checkfor all five skill filesquick_validate.py:Skill is valid!git diff --checkSummary by CodeRabbit
New Features
Documentation