Skip to content

Commit e9d77ce

Browse files
committed
feat: add with/without skill comparison
1 parent 2391dda commit e9d77ce

1 file changed

Lines changed: 13 additions & 4 deletions

File tree

doc/microsoft-foundry-skill-status.md

Lines changed: 13 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -36,6 +36,12 @@ The response should mention Foundry agent workflows such as scaffolding, local t
3636

3737
The Foundry skill performs consistently across different Copilot models. The benchmark tested Claude Opus 4.8, Claude Opus 4.6, Claude Sonnet 4.6, GPT-5.5, GPT-5.4, GPT-5.3-Codex, GPT-5 mini, and MAI-Code-1-Flash.
3838

39+
Benchmark prompt:
40+
41+
```text
42+
Create a Python hosted agent for B2B customer onboarding and deploy it to a new Foundry project. Use the Responses protocol. After it is done, run in locally to make sure it can run successfully; then deploy it to foundry and ensure it can respond to users correctly
43+
```
44+
3945
Representative results:
4046

4147
| Copilot model | Time | AI Credits |
@@ -44,11 +50,14 @@ Representative results:
4450
| Auto (GPT-5.3-Codex) | 9 min 30 s | 60 |
4551
| Free plan / cheapest (GPT-5 mini) | 11 min 30 s | 20 |
4652

47-
These results measure time cost and GitHub Copilot AI Credits for this golden path prompt:
53+
## Benchmark With and Without the Skill
4854

49-
```text
50-
Create a Python hosted agent for B2B customer onboarding and deploy it to a new Foundry project. Use the Responses protocol. After it is done, run in locally to make sure it can run successfully; then deploy it to foundry and ensure it can respond to users correctly
51-
```
55+
This comparison uses Sonnet 4.6 on the same benchmark prompt.
56+
57+
| Setup | Model | Time Cost | AIC |
58+
| --- | --- | ---: | ---: |
59+
| Without the skill | Sonnet 4.6 | 33 min 20 s | 410 |
60+
| With the skill | Sonnet 4.6 | 10 min 30 s (⬇️ 69%) | 100 (⬇️ 76%) |
5261

5362
## Run the Benchmark Yourself
5463

0 commit comments

Comments
 (0)