[prompt-analysis] Copilot PR Prompt Analysis - 2026-05-28

[github-actions[bot]]

Summary

Analysis Period: Last 30 days
Total PRs: 1,000 | Merged: 799 (80.5%) | Closed: 193 (19.5%) | Open: 8

Today's overall success rate is 80.5%, continuing a slight downward trend from ~81.7% on 2026-05-19.

Prompt Categories and Success Rates

Category	Total	Merged	Closed	Success Rate
Bug Fix	655	521	127	80%
Feature Addition	219	175	43	80%
Refactoring	34	29	5	85% ✅
Testing	28	24	4	86% ✅
Documentation	28	22	6	79%
Other	36	28	8	78%

Prompt Analysis

✅ Successful Prompt Patterns

Common characteristics in merged PRs:

• Average prompt length: 511 words (notably shorter than closed PRs)
• High-success categories: Testing (86%) and Refactoring (85%)
• Specific file references present in 64% of merged PRs
• Clear problem statement with targeted scope

Example successful prompts:

1. PR refactor: replace raw engine strings with typed constants.*Engine #35559: refactor: replace raw engine strings with typed constants.*Engine — Highly specific, scoped to named files (compiler_yaml_lookups.go, firewall.go) → Merged
2. PR Expose gh aw init --engine and skip Copilot-only scaffolding for non-Copilot engines #35542: Expose gh aw init --engine and skip Copilot-only scaffolding for non-Copilot engines — Concrete feature with clear before/after state → Merged
3. PR SPDD 2026-05-28: P0/P1/P2 spec work — implement SM-IS-01 10 KB limit (all transports), fix R-IMPL-040, add norms/vectors/safeguards #35528: SPDD 2026-05-28: P0/P1/P2 spec work — implement SM-IS-01 10 KB limit — Structured priority list with spec reference → Merged

❌ Unsuccessful Prompt Patterns

Common characteristics in closed PRs:

• Average prompt length: 668 words (31% longer than merged — verbosity ≠ success)
• WIP-tagged PRs: 29 closed vs 26 merged (essentially coin-flip)
• Generic openers like "Thanks for asking me to work on this..." signal low specificity
• Scope-creep patterns: migrating/restructuring many files at once

Example unsuccessful prompts:

1. PR [WIP] Migrate 52 workflows to shared reporting-otlp.md bundle #35541: [WIP] Migrate 52 workflows to shared reporting-otlp.md bundle — Massive scope, WIP-tagged → Closed
2. PR Refactor daily tracker workflows onto shared daily base bundles #35497: Refactor daily tracker workflows onto shared daily base bundles — Broad refactor across many files without targeted spec → Closed
3. PR [WIP] Fix healthcheck issue for awf-squid firewall sidecar #35488: [WIP] Fix healthcheck issue for awf-squid firewall sidecar — WIP tag + niche infrastructure change → Closed

Key Insights

• Conciseness wins: Merged PRs average 511 words vs 668 for closed (24% shorter). Over-specified or sprawling prompts correlate with lower success.
• WIP is a red flag: [WIP]-tagged PRs are essentially coin-flip (26 merged / 29 closed = 47% success vs 80.5% baseline). Avoid merging WIP PRs without resolving the WIP status first.
• Test & Refactor prompts outperform: These categories hit 85–86% success — likely because they are well-scoped and have clear acceptance criteria baked in.

Recommendations

1. DO: Keep prompts under ~500 words. Focus on one targeted change with clear before/after expectations.
2. DO: Reference specific file names and function names — it signals scope precision and helps the agent stay focused.
3. DO: Prefer test-writing or refactoring prompts for highest merge rates; frame bug fixes with reproduction context.
4. AVOID: WIP-tagged PRs in the merge queue — resolve WIP status before requesting review.
5. AVOID: Large-scale migration prompts ("migrate 52 workflows...") — break into smaller, independently mergeable units.

Historical Trends (Last 7 Days)

Date	PRs	Merged	Closed	Success Rate
2026-05-19	993	811	182	81.7%
2026-05-24	1,000	810	186	81.3%
2026-05-25	1,000	804	193	80.6%
2026-05-26	1,000	802	193	80.6%
2026-05-27	1,000	804	192	80.7%
2026-05-28	1,000	799	193	80.5%

Trend: Slight downward drift from ~81.7% to 80.5% over 9 days. The increase in closed PRs (from ~182 to 193) is persistent — this may reflect more complex/broader prompts being submitted recently.

---

Generated by Copilot PR Prompt Analysis · Run: §26602897691

Generated by 🔍 Copilot PR Prompt Pattern Analysis · sonnet46 1.1M · ◷

• expires on May 29, 2026, 9:25 PM UTC

[github-actions[bot]]

This discussion was automatically closed because it expired on 2026-05-29T21:25:03.661Z.

Closed by Workflow