fix: expand Network Mapping and Pattern Heuristics tables in agentic-workflow-designer SKILL.md#40249
Conversation
…workflow-designer SKILL.md Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
|
✅ PR Code Quality Reviewer completed the code quality review. |
|
✅ Test Quality Sentinel completed test quality analysis. No test files were added or modified in this PR. Test Quality Sentinel skipped. The PR only modifies documentation files: .github/skills/agentic-workflow-designer/SKILL.md and pkg/cli/data/agentic_workflow_designer_skill.md. |
|
✅ Design Decision Gate 🏗️ completed the design decision gate check. No ADR enforcement needed: PR #40249 does not have the 'implementation' label (has_implementation_label=false) and has only 34 new lines in business logic directories, which is at or below the 100-line threshold (requires_adr_by_default_volume=false). |
|
🧠 Matt Pocock Skills Reviewer has completed the skills-based review. ✅ |
There was a problem hiding this comment.
Pull request overview
Expands the agentic-workflow-designer skill’s routing tables so the designer can more reliably (a) recommend valid network.allowed ecosystem identifiers and (b) route user intents to all documented workflow patterns.
Changes:
- Expanded Network Mapping to include many additional ecosystem identifiers (e.g.,
github,github-actions,containers,playwright, etc.). - Expanded Pattern Heuristics to cover all patterns currently indexed in
.github/aw/patterns.md. - Updated both the source skill and its embedded CLI copy in sync.
Show a summary per file
| File | Description |
|---|---|
.github/skills/agentic-workflow-designer/SKILL.md |
Adds additional Network Mapping and Pattern Heuristics entries used by the workflow designer. |
pkg/cli/data/agentic_workflow_designer_skill.md |
Keeps the CLI-embedded copy of the designer skill in sync with the expanded tables. |
Copilot's findings
Tip
Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
- Files reviewed: 2/2 changed files
- Comments generated: 6
| | "uses Elixir / Hex packages" | include `elixir` in `network.allowed` | | ||
| | "uses Bazel build" | include `bazel` in `network.allowed` | | ||
| | "uses R / CRAN packages" | include `r` in `network.allowed` | | ||
| | "no external access" | `network.allowed: [defaults]` (or `[]` if explicitly zero network) | |
| | "coordinate multiple sub-agents" | `Orchestration` | | ||
| | "manage project board items" | `ProjectOps` | | ||
| | "research, plan, and assign issues" | `ResearchPlanAssignOps` | | ||
| | "self-correcting / retry on failure" | `CorrectionOps` | |
| | "write a spec before implementing" | `SpecOps` | | ||
| | "A/B test workflow variants" | `TrialOps` | | ||
| | "process items from a queue" | `WorkQueueOps` | | ||
| | "deterministic, no LLM needed" | `DeterministicOps` | |
| | "uses Elixir / Hex packages" | include `elixir` in `network.allowed` | | ||
| | "uses Bazel build" | include `bazel` in `network.allowed` | | ||
| | "uses R / CRAN packages" | include `r` in `network.allowed` | | ||
| | "no external access" | `network.allowed: [defaults]` (or `[]` if explicitly zero network) | |
| | "coordinate multiple sub-agents" | `Orchestration` | | ||
| | "manage project board items" | `ProjectOps` | | ||
| | "research, plan, and assign issues" | `ResearchPlanAssignOps` | | ||
| | "self-correcting / retry on failure" | `CorrectionOps` | |
| | "write a spec before implementing" | `SpecOps` | | ||
| | "A/B test workflow variants" | `TrialOps` | | ||
| | "process items from a queue" | `WorkQueueOps` | | ||
| | "deterministic, no LLM needed" | `DeterministicOps` | |
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
There was a problem hiding this comment.
Skills-Based Review 🧠
Applied /grill-with-docs — requesting changes on two trigger-phrase accuracy bugs that would cause incorrect pattern routing.
📋 Key Themes & Highlights
Blocking Issues (2)
CorrectionOpstrigger phrase —"self-correcting / retry on failure"routes users to a human-feedback learning pattern when they likely want retry/error-recovery behavior. These are fundamentally different concepts.DeterministicOpstrigger phrase —"deterministic, no LLM needed"contradicts the pattern which explicitly includes agentic (LLM) analysis as its second step.
Non-blocking Observations (4)
TrialOps—"A/B test workflow variants"implies concurrent comparison; the pattern is about sequential pre-production validation in isolated repos.SpecOps—"write a spec before implementing"misses the spec-propagation lifecycle that defines this pattern.SideRepoOps—"fork repo"conflates git forks with companion/satellite repos.- Missing network identifiers — 14 identifiers from
network.md(e.g.node-cdns,dev-tools,chrome,powershell,kotlin) are still absent. Either add them or clarify in the PR description that coverage is intentionally curated.
Positive Highlights
- ✅ Excellent coverage expansion: Pattern Heuristics now covers all 19 patterns from
patterns.md(was 6) - ✅ Both canonical SKILL.md and the embedded CLI copy are updated in sync
- ✅ Network Mapping additions for the most common ecosystems (go, node, python, rust, java, containers, etc.) are accurate and well-phrased
- ✅ Purely additive change — no existing routing is disturbed
🧠 Reviewed using Matt Pocock's skills by Matt Pocock Skills Reviewer · 71.1 AIC · ⌖ 7.93 AIC · ⊞ 6.9K
| | "coordinate multiple sub-agents" | `Orchestration` | | ||
| | "manage project board items" | `ProjectOps` | | ||
| | "research, plan, and assign issues" | `ResearchPlanAssignOps` | | ||
| | "self-correcting / retry on failure" | `CorrectionOps` | |
There was a problem hiding this comment.
[/grill-with-docs] The trigger phrase "self-correcting / retry on failure" does not match CorrectionOps.
CorrectionOps is defined in patterns.md as: "improve workflow behavior from trusted human corrections without retraining the model" — it is a human-feedback learning loop, not an automated retry mechanism.
A user describing "self-correcting / retry on failure" behaviour would be incorrectly routed to CorrectionOps.
💡 Suggested fix
Align the phrase with the actual pattern intent:
-| "self-correcting / retry on failure" | `CorrectionOps` |
+| "learn from human corrections without retraining" | `CorrectionOps` |If retry-on-failure routing is also needed, add a separate row pointing to an appropriate pattern (e.g. BatchOps with retry logic).
| | "write a spec before implementing" | `SpecOps` | | ||
| | "A/B test workflow variants" | `TrialOps` | | ||
| | "process items from a queue" | `WorkQueueOps` | | ||
| | "deterministic, no LLM needed" | `DeterministicOps` | |
There was a problem hiding this comment.
[/grill-with-docs] The trigger phrase "deterministic, no LLM needed" contradicts the DeterministicOps pattern definition.
DeterministicOps is defined as: "deterministic data collection steps followed by agentic analysis and reporting" — the second step explicitly uses an LLM. Routing a user who wants a fully LLM-free workflow to DeterministicOps will produce a design that doesn't match their intent.
💡 Suggested fix
-| "deterministic, no LLM needed" | `DeterministicOps` |
+| "deterministic data collection + agentic analysis" | `DeterministicOps` |For users who truly want no LLM, the correct answer is a plain GitHub Actions workflow — not a pattern in this guide at all.
| | "self-correcting / retry on failure" | `CorrectionOps` | | ||
| | "run in a side/fork repo" | `SideRepoOps` | | ||
| | "write a spec before implementing" | `SpecOps` | | ||
| | "A/B test workflow variants" | `TrialOps` | |
There was a problem hiding this comment.
[/grill-with-docs] "A/B test workflow variants" is an imprecise match for TrialOps.
TrialOps is defined as "isolated trial repositories to validate workflows before production rollout" — it's about pre-production safety validation in isolated repos, not running simultaneous A/B comparisons. The phrase "A/B test" implies concurrent variant testing, which isn't what this pattern provides.
💡 Suggested fix
-| "A/B test workflow variants" | `TrialOps` |
+| "validate in an isolated trial repo before production" | `TrialOps` || | "research, plan, and assign issues" | `ResearchPlanAssignOps` | | ||
| | "self-correcting / retry on failure" | `CorrectionOps` | | ||
| | "run in a side/fork repo" | `SideRepoOps` | | ||
| | "write a spec before implementing" | `SpecOps` | |
There was a problem hiding this comment.
[/grill-with-docs] "write a spec before implementing" is too narrow for SpecOps.
SpecOps is defined as "maintain formal specifications and propagate spec updates to consuming implementations" — it's a durable spec-lifecycle pattern, not just spec-first development. A user wanting spec-driven design for an existing system (updating a spec and pushing changes downstream) won't match this phrase.
💡 Suggested fix
-| "write a spec before implementing" | `SpecOps` |
+| "maintain a formal spec and propagate updates to implementations" | `SpecOps` || | "uses Deno or JSR packages" | include `deno` in `network.allowed` | | ||
| | "uses Elixir / Hex packages" | include `elixir` in `network.allowed` | | ||
| | "uses Bazel build" | include `bazel` in `network.allowed` | | ||
| | "uses R / CRAN packages" | include `r` in `network.allowed` | |
There was a problem hiding this comment.
[/grill-with-docs] 14 ecosystem identifiers from network.md are still absent from this table after the expansion.
The PR description claims to cover "all high-frequency ecosystem identifiers plus additional long-tail ones", but network.md lists 39 identifiers total and the SKILL.md now covers 26 — leaving these still unmapped:
clojure, julia, kotlin, lua, node-cdns, ocaml, powershell, scala, zig, dev-tools, chrome, latex, lean, python-native
Some of these (node-cdns, dev-tools, chrome, powershell, kotlin) are plausible choices in CI/CD workflows.
💡 Suggested additions
High-priority missing entries:
| User says... | Maps to |
|---|---|
| "uses CDN / jsdelivr / unpkg" | include node-cdns in network.allowed |
| "uses Renovate / Codecov / shields.io" | include dev-tools in network.allowed |
| "runs Chrome or Chromium" | include chrome in network.allowed |
| "uses PowerShell Gallery" | include powershell in network.allowed |
| "uses Kotlin / JetBrains packages" | include kotlin in network.allowed |
Or, if the intent is intentional curation rather than exhaustive coverage, update the PR description to say so.
| | "manage project board items" | `ProjectOps` | | ||
| | "research, plan, and assign issues" | `ResearchPlanAssignOps` | | ||
| | "self-correcting / retry on failure" | `CorrectionOps` | | ||
| | "run in a side/fork repo" | `SideRepoOps` | |
There was a problem hiding this comment.
[/grill-with-docs] "run in a side/fork repo" conflates two distinct concepts.
SideRepoOps is defined as "low-friction reporting/automation from a side repository targeting a primary repository" — a side repo is a satellite/companion repository, not a git fork. Saying "fork repo" risks routing users who want to work with forked repositories (a common GitHub concept) into a pattern that's actually about companion automation repos.
💡 Suggested fix
-| "run in a side/fork repo" | `SideRepoOps` |
+| "run automation from a companion/satellite repository" | `SideRepoOps` |
There was a problem hiding this comment.
REQUEST_CHANGES — good coverage expansion, but three of the newly added heuristics map to wrong/misleading values that will cause designer agents to produce incorrect output.
### Blocking issues in this PR's added lines
1. Orchestration is not a pattern name (line 210) — both doc files declare title: OrchestratorOps. Inline comment posted.
2. TrialOps trigger phrase is wrong (line 216) — TrialOps is isolated trial-repo validation, not A/B testing. There is a separate A/B Experiments feature. Inline comment posted.
3. CorrectionOps trigger phrase is wrong (line 213) — already flagged in a prior review thread. CorrectionOps is a human-correction learning loop, not a retry-on-failure mechanism. This issue remains unresolved.
4. DeterministicOps trigger phrase is wrong (line 218) — already flagged in a prior review thread. DeterministicOps explicitly combines deterministic steps with AI agent reasoning; calling it "no LLM needed" contradicts its own documentation. This issue remains unresolved.
Fix all four before merging. The two-file sync (SKILL.md ↔ pkg/cli/data/agentic_workflow_designer_skill.md) must be applied consistently.
🔎 Code quality review by PR Code Quality Reviewer · 85.6 AIC · ⌖ 7.13 AIC · ⊞ 5.1K
| | "run manually with input parameters" | `DispatchOps` | | ||
| | "apply a label-based workflow" | `LabelOps` | | ||
| | "operate across multiple repositories" | `MultiRepoOps` | | ||
| | "coordinate multiple sub-agents" | `Orchestration` | |
There was a problem hiding this comment.
Wrong pattern name: Orchestration does not exist — the canonical name is OrchestratorOps and will not match any documentation link or search.
💡 Suggested fix
Both pattern doc files (orchestration.md and orchestrator-ops.md) declare title: OrchestratorOps. Using Orchestration here means every agent following this heuristic will recommend a pattern name that users cannot find in the docs.
-| "coordinate multiple sub-agents" | `Orchestration` |
+| "coordinate multiple sub-agents" | `OrchestratorOps` |Apply the same fix to pkg/cli/data/agentic_workflow_designer_skill.md.
| | "self-correcting / retry on failure" | `CorrectionOps` | | ||
| | "run in a side/fork repo" | `SideRepoOps` | | ||
| | "write a spec before implementing" | `SpecOps` | | ||
| | "A/B test workflow variants" | `TrialOps` | |
There was a problem hiding this comment.
Trigger phrase mismatches pattern purpose: TrialOps is about validating workflows in isolated trial repositories before deployment — not A/B testing variants.
💡 Details and suggested fix
The TrialOps docs describe a gh aw trial command that spins up a temporary private repo to safely validate one workflow before shipping it. There is also a separate A/B Experiments feature (experimental/experiments.md) for actual variant comparison.
A user asking to "A/B test workflow variants" will be steered to TrialOps when they likely need A/B Experiments or simply want to run two different workflows side-by-side.
Suggested trigger phrase that actually describes TrialOps:
-| "A/B test workflow variants" | `TrialOps` |
+| "validate a workflow in an isolated test repo before deploying" | `TrialOps` |Apply the same fix to pkg/cli/data/agentic_workflow_designer_skill.md.
The
agentic-workflow-designerSKILL.md had significant coverage gaps: its Network Mapping table covered only 4 of 35 ecosystem identifiers fromnetwork.md, and its Pattern Heuristics table covered only 6 of 19 patterns frompatterns.md, leaving the designer unable to route 13 patterns or recommend most network identifiers.Network Mapping (
### Network Mapping)Expanded from 4 → 26 entries to cover all high-frequency ecosystem identifiers plus additional long-tail ones:
github,github-actions,ruby,rust,dotnet,java,containers,playwright,linux-distros,terraform,localswift,php,dart,haskell,perl,fonts,deno,elixir,bazel,rPattern Heuristics (
### Pattern Heuristics)Expanded from 6 → 19 entries, adding all 13 previously unmapped patterns with user-facing phrases aligned to the
patterns.mdrouting rules:LabelOpsMultiRepoOpsOrchestrationProjectOpsResearchPlanAssignOpsCorrectionOpsSideRepoOpsSpecOpsTrialOpsWorkQueueOpsDeterministicOpsCentralRepoOpsMonitoring with ProjectsBoth
.github/skills/agentic-workflow-designer/SKILL.mdand its embedded copypkg/cli/data/agentic_workflow_designer_skill.mdare updated in sync.