Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,7 @@ Additional submissions that are not directly comparable to the main leaderboard
| Agent | LLM(s) used | Low == Lite (%) | Medium (%) | High (%) | All (%) | Running Time (hours) | Date | Notes | Source Code Available | Grading Reports Available |
|-------|-------------|-----------------|------------|----------|---------|----------------------|------|-------|----------------------|---------------------------|
| [Disarray](https://disarray.ai) | Ensemble (Claude-Opus-4.5, Claude-Sonnet-4.5, GPT-5.2-Codex, Gemini-3-Pro-Preview) | 90.91 ± 0.00 | 72.81 ± 0.88 | 71.11 ± 2.22 | 77.78 ± 0.44 | 24 | 2026-02-03 | [Test-set feedback](https://github.com/openai/mle-bench/pull/118) | X ||
| [AiScientist](https://github.com/AweAI-Team/AiScientist)<br>(AweAI Team) | GLM-5 | 81.82 ± 0.00 | N/A | N/A | N/A | 24 | 2026-04-15 | Lite only (22-task low split); no held-out test-set feedback |||
| [LoongFlow](https://github.com/baidu-baige/LoongFlow) | Gemini-3-Flash-Preview | 77.27 ± 0.0[^3] | 63.15 ± 1.51[^3] | 40.0 ± 0.00[^3] | 62.66 ± 0.76[^3] | 24 | 2026-02-09 | [Test-set feedback](https://github.com/openai/mle-bench/pull/119) |||

[^2]: With some light assistance from an ensemble of models including
Expand Down
3 changes: 2 additions & 1 deletion runs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -60,4 +60,5 @@ table below.
| MLEvolve | Gemini-3-Pro-preview, 12 hours, 21 vCPUs, 234GB of RAM, and 1 H200 GPU |
| MARS | CAIR MARS, 24 hours, 12 vCPUs, 220GB of RAM, and 1 A100-40GB GPU |
| MARS+ | CAIR MARS+, 24 hours, 48 vCPUs, 220GB of RAM, and 2 × H100 GPUs |
| AIBuildAI | Claude-Opus-4.6, 24 hours, 24 vCPUs, 256GB of RAM, and 1 A100 GPU |
| AIBuildAI | Claude-Opus-4.6, 24 hours, 24 vCPUs, 256GB of RAM, and 1 A100 GPU |
| AiScientist-GLM5-Lite-24h | GLM-5, 24 hours, 16 vCPUs, and 1 H20 GPU |
3 changes: 3 additions & 0 deletions runs/aiscientist_glm5_lite_group1/grading_report_group_1.json
Git LFS file not shown
3 changes: 3 additions & 0 deletions runs/aiscientist_glm5_lite_group2/grading_report_group_2.json
Git LFS file not shown
3 changes: 3 additions & 0 deletions runs/aiscientist_glm5_lite_group3/grading_report_group_3.json
Git LFS file not shown
4 changes: 2 additions & 2 deletions runs/run_group_experiments.csv
Git LFS file not shown