{Agent} Add AI-powered custom code agent - #33760
Open
Qi Pan (Pan-Qi) wants to merge 7 commits into
Open
Conversation
|
Hi Qi Pan (@Pan-Qi), |
|
Azure Pipelines: There may be pipelines that require an authorized user to comment /azp run to run. |
Collaborator
|
Thank you for your contribution! We will review the pull request and get back to you soon. |
Contributor
Live test skipped⏭️ Skipping the live test for this revision because no new test file was added ( The live-test pipeline runs only the test files a PR changes, so there is nothing to execute for this commit. This is informational — a regression test is encouraged where it makes sense, but not required. If a test file is added in a later commit, the live test will run automatically. |
![x-engineering-agent[bot]](https://avatars.githubusercontent.com/in/4205534?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Automated Review Summary — PR #33760
Live test: ⚪ Skipped (no live-testable command module test files changed)
CI status: ❌ 4 checks failed, 88 passed
Failed checks:
- Build — Build #20260818.16 failed
- Test Homebrew Formula — failed
- macOS | Build CLI Intel — failed
- Codegen Coverage — failed
The core Build failure is likely the root cause and worth investigating first;
the Homebrew/macOS build failures may be downstream of it.
This PR was reviewed on request (Azure Client Tools Agent Requested); a
human maintainer should follow up since automated Copilot re-triggering is not
used for human-authored PRs.
x-engineering-agent
Bot
added
azure-client-tools-agent
and removed
Azure Client Tools Agent Requested
labels
Qi Pan (Pan-Qi)
marked this pull request as ready for review
August 24, 2026 07:37
Contributor
There was a problem hiding this comment.
Pull request overview
Adds an AI-assisted workflow for Azure CLI custom code authoring by introducing a @custom-code-agent, OpenSpec configuration scaffolding, and a set of OpenSpec-related prompts/skills to guide proposal → implementation → sync → archive flows.
Changes:
- Add OpenSpec scaffolding (
openspec/config.yaml, keepalive directories, and gitignore rules) to support spec-driven change planning. - Introduce documentation and Copilot artifacts (agent + instructions + skills + prompts) for AI-assisted custom code authoring workflows.
- Add common skills/templates to route and generate Azure CLI custom code (AAZ subclass, legacy SDK-backed, legacy non-SDK) and a help-to-live-test scaffold.
Reviewed changes
Copilot reviewed 23 out of 26 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| openspec/specs/.gitkeep | Keep openspec/specs/ tracked until real specs are added. |
| openspec/config.yaml | Add OpenSpec repository context and spec-driven workflow rules. |
| openspec/changes/.gitkeep | Keep openspec/changes/ tracked as a workspace root. |
| doc/ai_custom_code.md | Document how to use @custom-code-agent vs OpenSpec workflows. |
| .gitignore | Ignore local OpenSpec change workspace contents while keeping .gitkeep. |
| .github/skills/openspec-update-change/SKILL.md | Skill definition for updating existing OpenSpec change artifacts. |
| .github/skills/openspec-sync-specs/SKILL.md | Skill definition for syncing delta specs into main specs. |
| .github/skills/openspec-propose/SKILL.md | Skill definition for creating a new OpenSpec change and artifacts. |
| .github/skills/openspec-explore/SKILL.md | Skill definition for exploration mode (investigation/planning without implementation). |
| .github/skills/openspec-archive-change/SKILL.md | Skill definition for archiving an OpenSpec change, optionally syncing specs. |
| .github/skills/openspec-apply-change/SKILL.md | Skill definition for implementing tasks from an OpenSpec change. |
| .github/skills/common-live-test-from-help/SKILL.md | Skill definition for deriving live test scaffolding from help examples. |
| .github/skills/common-live-test-from-help/references/live-test-template.md | Template for generated help-driven live test structure. |
| .github/skills/common-legacy-sdk-backed-custom-code-generation/SKILL.md | Guidance/templates for SDK-backed custom command handlers. |
| .github/skills/common-legacy-non-sdk-custom-code-generation/SKILL.md | Guidance/templates for non-SDK custom workflows (tools/subprocess/raw HTTP/etc.). |
| .github/skills/common-custom-code-router/SKILL.md | Routing classifier to choose exactly one custom-code authoring route. |
| .github/skills/common-aaz-custom-code-generation/SKILL.md | Guidance/templates for AAZ subclass customization workflow. |
| .github/prompts/opsx-update.prompt.md | Prompt to revise an OpenSpec change’s planning artifacts. |
| .github/prompts/opsx-sync.prompt.md | Prompt to sync delta specs from a change into main specs. |
| .github/prompts/opsx-propose.prompt.md | Prompt to propose a new change and generate artifacts. |
| .github/prompts/opsx-explore.prompt.md | Prompt to enter explore mode for investigation/planning. |
| .github/prompts/opsx-archive.prompt.md | Prompt to archive a completed change. |
| .github/prompts/opsx-apply.prompt.md | Prompt to implement tasks from a change. |
| .github/instructions/custom-code.instructions.md | File-scoped Copilot guidance for editing CLI module custom code. |
| .github/instructions/agent-safety.instructions.md | Global safety/scope rules for all agents. |
| .github/agents/custom-code.agent.md | Define the custom-code-agent behavior and the localization entry gate. |
Suppressed comments (4)
.github/skills/common-legacy-non-sdk-custom-code-generation/SKILL.md:128
- This template imports
knack.util.CLIError, which is deprecated perdoc/error_handling_guidelines.md. Use anazure.cli.core.azclierrortype here so generated code follows current repo guidance.
from knack.util import CLIError
.github/skills/common-legacy-non-sdk-custom-code-generation/SKILL.md:133
- This template raises
CLIError, which is deprecated perdoc/error_handling_guidelines.md. Update the template to raise anazure.cli.core.azclierrortype so generated code follows current repo guidance.
raise CLIError(stderr.decode().strip() or "Command failed.")
.github/skills/common-aaz-custom-code-generation/SKILL.md:189
- This template raises
CLIErrorfor an invalid resource ID format, but the repo guidance is to useazure.cli.core.azclierrortypes.InvalidArgumentValueErroris a better fit here and keeps templates aligned with the error handling guidelines.
raise CLIError(f"Invalid resource ID format: {resource_uri}")
.github/skills/common-aaz-custom-code-generation/SKILL.md:200
- This template raises
CLIErrorwhen the checked resource already exists. Perdoc/error_handling_guidelines.md, templates should useazure.cli.core.azclierrortypes;ValidationErrorfits a pre-operation state check and lets you put the actionable guidance inrecommendation.
raise CLIError("The resource already exists. Use update to modify it or delete it before creating a new one.")
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
| - If a new command is added, wire it in commands.py and define args in _params.py. | ||
| - Keep return shapes compatible with existing CLI output patterns. | ||
| - Keep help/examples aligned with command behavior. | ||
| - Prefer concise error messages through CLIError when user input is invalid. |
Comment on lines
+118
to
+122
| from knack.util import CLIError | ||
|
|
||
| command = '<tool>' | ||
| if not shutil.which(command): | ||
| raise CLIError("'<tool>' is required for this command. Install it and try again.") |
| def _check_<condition>(ctx): | ||
| from azure.cli.core.azclierror import HTTPError | ||
| from azure.cli.core.util import send_raw_request | ||
| from knack.util import CLIError |
Qi Pan (Pan-Qi)
changed the title
Qi Pan (Pan-Qi)
changed the title
Aditya Pujara (a0x1ab)
added
X Engineering Agent Reviewed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
🤖 PR Validation — ️✔️ All clear
Related command
Description
Testing Guide
History Notes
[Component Name 1] BREAKING CHANGE:
az command a: Make some customer-facing breaking change[Component Name 2]
az command b: Add some customer-facing featureThis checklist is used to make sure that common guidelines for a pull request are followed.
The PR title and description has followed the guideline in Submitting Pull Requests.
I adhere to the Command Guidelines.
I adhere to the Error Handling Guidelines.