Skip to content

Commit 2391dda

Browse files
authored
Merge pull request #504 from microsoft/anchenyi/add-skill-docs
feat: add Microsoft Foundry skill status and benchmark doc
2 parents d2264d6 + 9a4c31b commit 2391dda

1 file changed

Lines changed: 81 additions & 0 deletions

File tree

Lines changed: 81 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,81 @@
1+
# Microsoft Foundry Skill Status and Benchmark
2+
3+
Status and benchmark results for the `microsoft-foundry` skill, with prompts you can use to reproduce the same tests.
4+
5+
The `microsoft-foundry` skill is used by Copilot-assisted agent development in Microsoft Foundry Toolkit.
6+
7+
**Source code:** [`microsoft/azure-skills/skills/microsoft-foundry`](https://github.com/microsoft/azure-skills/tree/main/skills/microsoft-foundry)
8+
9+
**Quick links:** [Install the skill](#install-the-skill) | [Current benchmark](#current-benchmark) | [Run the benchmark yourself](#run-the-benchmark-yourself)
10+
11+
## Install the Skill
12+
13+
Install from `microsoft/azure-skills`:
14+
15+
```bash
16+
npx skills add https://github.com/microsoft/azure-skills --skill microsoft-foundry
17+
```
18+
19+
The skill installs to:
20+
21+
```text
22+
~/.agent/skills/microsoft-foundry
23+
```
24+
25+
## Verify the Installation
26+
27+
Start a fresh GitHub Copilot CLI session and ask:
28+
29+
```text
30+
Use the microsoft-foundry skill. What can you help me do?
31+
```
32+
33+
The response should mention Foundry agent workflows such as scaffolding, local testing, deployment, invocation, and evaluation.
34+
35+
## Current Benchmark
36+
37+
The Foundry skill performs consistently across different Copilot models. The benchmark tested Claude Opus 4.8, Claude Opus 4.6, Claude Sonnet 4.6, GPT-5.5, GPT-5.4, GPT-5.3-Codex, GPT-5 mini, and MAI-Code-1-Flash.
38+
39+
Representative results:
40+
41+
| Copilot model | Time | AI Credits |
42+
| --- | ---: | ---: |
43+
| Sonnet 4.6 | 10 min 30 s | 100 |
44+
| Auto (GPT-5.3-Codex) | 9 min 30 s | 60 |
45+
| Free plan / cheapest (GPT-5 mini) | 11 min 30 s | 20 |
46+
47+
These results measure time cost and GitHub Copilot AI Credits for this golden path prompt:
48+
49+
```text
50+
Create a Python hosted agent for B2B customer onboarding and deploy it to a new Foundry project. Use the Responses protocol. After it is done, run in locally to make sure it can run successfully; then deploy it to foundry and ensure it can respond to users correctly
51+
```
52+
53+
## Run the Benchmark Yourself
54+
55+
### Prerequisites
56+
57+
- Azure subscription
58+
- Azure CLI (`az`)
59+
- Azure Developer CLI (`azd`)
60+
- Python
61+
62+
### End-to-end Testing
63+
64+
Use the golden path prompt:
65+
66+
```text
67+
Create a Python hosted agent for B2B customer onboarding and deploy it to a new Foundry project. Use the Responses protocol. After it is done, run in locally to make sure it can run successfully; then deploy it to foundry and ensure it can respond to users correctly
68+
```
69+
70+
### Stage-by-stage Testing
71+
72+
Use these prompts when you want to isolate one scenario instead of measuring the full end-to-end path.
73+
74+
| Scenario | Prompt |
75+
| --- | --- |
76+
| 1a Configuration and scaffolding (greenfield) | `Create a Python hosted agent for B2B customer onboarding and set up whatever I need.` |
77+
| 1b Configuration and scaffolding (brownfield) | `I have an existing agent built with the OpenAI SDK. I want to deploy it as a Foundry hosted agent. What do I need to change?` |
78+
| 2 Local testing | `Help me run this agent locally and test it before I deploy.` |
79+
| 3 Deployment | `I'm happy with it locally. Deploy this to Foundry.` |
80+
| 4 Direct Toolbox iteration | `Add a WorkIQ tool to my deployed Foundry agent and redeploy.` |
81+
| 4-iter Casual iteration | `Can my agent also look up emails and meetings from my Outlook to help with onboarding?` |

0 commit comments

Comments
 (0)