Skip to content

Commit 597c021

Browse files
authored
Update exact match metric for GPT-4o results
1 parent d615a9c commit 597c021

1 file changed

Lines changed: 1 addition & 1 deletion

File tree

  • reproduction-results/gptmodel/results

reproduction-results/gptmodel/results/README.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -56,7 +56,7 @@ The table below compares this frontier-model run against the FLAMES-CodeLlama
5656

5757
| Metric | FLAMES-CodeLlama | GPT-4o (this run) |
5858
|---|---|---|
59-
| Exact match (syntactic, FLAMES rules) | 2/28 — **7.1%** | 10/30**33.3%** |
59+
| Exact match (syntactic, FLAMES rules) | 2/28 — **7.1%** | 9/28**32.1%** |
6060
| Garbage output (context truncation) | 24/28 — **85.7%** | 0 |
6161
| Empty (token limit) | 0 | 1/31 |
6262

0 commit comments

Comments
 (0)