|
| 1 | +# Microsoft Foundry Skill Status and Benchmark |
| 2 | + |
| 3 | +Status and benchmark results for the `microsoft-foundry` skill, with prompts you can use to reproduce the same tests. |
| 4 | + |
| 5 | +The `microsoft-foundry` skill is used by Copilot-assisted agent development in Microsoft Foundry Toolkit. |
| 6 | + |
| 7 | +**Source code:** [`microsoft/azure-skills/skills/microsoft-foundry`](https://github.com/microsoft/azure-skills/tree/main/skills/microsoft-foundry) |
| 8 | + |
| 9 | +**Quick links:** [Install the skill](#install-the-skill) | [Current benchmark](#current-benchmark) | [Run the benchmark yourself](#run-the-benchmark-yourself) |
| 10 | + |
| 11 | +## Install the Skill |
| 12 | + |
| 13 | +Install from `microsoft/azure-skills`: |
| 14 | + |
| 15 | +```bash |
| 16 | +npx skills add https://github.com/microsoft/azure-skills --skill microsoft-foundry |
| 17 | +``` |
| 18 | + |
| 19 | +The skill installs to: |
| 20 | + |
| 21 | +```text |
| 22 | +~/.agent/skills/microsoft-foundry |
| 23 | +``` |
| 24 | + |
| 25 | +## Verify the Installation |
| 26 | + |
| 27 | +Start a fresh GitHub Copilot CLI session and ask: |
| 28 | + |
| 29 | +```text |
| 30 | +Use the microsoft-foundry skill. What can you help me do? |
| 31 | +``` |
| 32 | + |
| 33 | +The response should mention Foundry agent workflows such as scaffolding, local testing, deployment, invocation, and evaluation. |
| 34 | + |
| 35 | +## Current Benchmark |
| 36 | + |
| 37 | +The Foundry skill performs consistently across different Copilot models. The benchmark tested Claude Opus 4.8, Claude Opus 4.6, Claude Sonnet 4.6, GPT-5.5, GPT-5.4, GPT-5.3-Codex, GPT-5 mini, and MAI-Code-1-Flash. |
| 38 | + |
| 39 | +Representative results: |
| 40 | + |
| 41 | +| Copilot model | Time | AI Credits | |
| 42 | +| --- | ---: | ---: | |
| 43 | +| Sonnet 4.6 | 10 min 30 s | 100 | |
| 44 | +| Auto (GPT-5.3-Codex) | 9 min 30 s | 60 | |
| 45 | +| Free plan / cheapest (GPT-5 mini) | 11 min 30 s | 20 | |
| 46 | + |
| 47 | +These results measure time cost and GitHub Copilot AI Credits for this golden path prompt: |
| 48 | + |
| 49 | +```text |
| 50 | +Create a Python hosted agent for B2B customer onboarding and deploy it to a new Foundry project. Use the Responses protocol. After it is done, run in locally to make sure it can run successfully; then deploy it to foundry and ensure it can respond to users correctly |
| 51 | +``` |
| 52 | + |
| 53 | +## Run the Benchmark Yourself |
| 54 | + |
| 55 | +### Prerequisites |
| 56 | + |
| 57 | +- Azure subscription |
| 58 | +- Azure CLI (`az`) |
| 59 | +- Azure Developer CLI (`azd`) |
| 60 | +- Python |
| 61 | + |
| 62 | +### End-to-end Testing |
| 63 | + |
| 64 | +Use the golden path prompt: |
| 65 | + |
| 66 | +```text |
| 67 | +Create a Python hosted agent for B2B customer onboarding and deploy it to a new Foundry project. Use the Responses protocol. After it is done, run in locally to make sure it can run successfully; then deploy it to foundry and ensure it can respond to users correctly |
| 68 | +``` |
| 69 | + |
| 70 | +### Stage-by-stage Testing |
| 71 | + |
| 72 | +Use these prompts when you want to isolate one scenario instead of measuring the full end-to-end path. |
| 73 | + |
| 74 | +| Scenario | Prompt | |
| 75 | +| --- | --- | |
| 76 | +| 1a Configuration and scaffolding (greenfield) | `Create a Python hosted agent for B2B customer onboarding and set up whatever I need.` | |
| 77 | +| 1b Configuration and scaffolding (brownfield) | `I have an existing agent built with the OpenAI SDK. I want to deploy it as a Foundry hosted agent. What do I need to change?` | |
| 78 | +| 2 Local testing | `Help me run this agent locally and test it before I deploy.` | |
| 79 | +| 3 Deployment | `I'm happy with it locally. Deploy this to Foundry.` | |
| 80 | +| 4 Direct Toolbox iteration | `Add a WorkIQ tool to my deployed Foundry agent and redeploy.` | |
| 81 | +| 4-iter Casual iteration | `Can my agent also look up emails and meetings from my Outlook to help with onboarding?` | |
0 commit comments