You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
**ClawBench is an open-source benchmark that evaluates AI browser agents on everyday online tasks — booking travel, ordering food, applying for jobs, managing email — across live websites. V1 lives in `test-cases/v1/` with 153 tasks across 144 websites; V2 lives in `test-cases/v2/` with 130 tasks. It measures end-to-end task success with a 5-layer recording pipeline and an agentic evaluator that compares each run against human references. Top score to date: 33.3%.**
ClawBench ships **three** Hugging Face datasets — task definitions plus full execution traces for V1 and V2. All open, downloadable in one command. The benchmark itself is also mirrored on **TIGER-Lab** for visibility.
176
176
@@ -200,13 +200,13 @@ ClawBench ships **three** Hugging Face datasets — task definitions plus full e
Point your coding agent (Claude Code, Cursor, Copilot, etc.) at [`AGENTS.md`](AGENTS.md) and prompt away.
206
206
207
207
<br/>
208
208
209
-
# <imgsrc="static/icons/person.svg"width="28"height="28"> Human Quick Start
209
+
# <imgsrc="assets/icons/person.svg"width="28"height="28"> Human Quick Start
210
210
211
211
Install ClawBench from PyPI for normal use:
212
212
@@ -348,7 +348,7 @@ This path gives you live-reload on ``src/``, ``src/clawbench/runtime/chrome-exte
348
348
349
349
<br/>
350
350
351
-
# <imgsrc="static/icons/check-double.svg"width="28"height="28"> Reproduce the leaderboard
351
+
# <imgsrc="assets/icons/check-double.svg"width="28"height="28"> Reproduce the leaderboard
352
352
353
353
> **Our scores are stable**: two independent runs of the same model under the same judge (`deepseek/deepseek-v4-pro`, lenient rubric) reproduce Intercepted and Reward within ±2 pp on the V2 130-task corpus.
354
354
@@ -386,7 +386,7 @@ For `deepseek-v4-flash:free × hermes × v2`, the published row is **Intercepted
**New here? Run this first.**[`test-cases/v1-lite/`](test-cases/v1-lite/) is a **20-task curated subset** of the V1 153-task corpus, selected for household-name sites, real-world relevance, difficulty, and category diversity. It matches the 20-tasks-per-source convention of [browser-use/benchmark](https://github.com/browser-use/benchmark) and gives you a credible signal at a fraction of the full-benchmark cost.
392
392
@@ -396,13 +396,13 @@ The Lite suite is a first-class task directory: run it with `--cases-suite v1-li
0 commit comments