Skip to content

Commit eaa5d4c

Browse files
authored
Merge pull request #203 from TIGER-AI-Lab/chore/202-static-to-asset-
Cleanup repo root #202
2 parents 44b4ea2 + db53d64 commit eaa5d4c

27 files changed

Lines changed: 88 additions & 108 deletions

.dockerignore

Lines changed: 0 additions & 14 deletions
This file was deleted.

.gitignore

Lines changed: 2 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -13,13 +13,5 @@ build/
1313
.idea/
1414
.vscode/
1515
*.log
16-
17-
# ClawBench-Lite upstream adapter build artifacts
18-
clawbench_lite_v1.plaintext.json
19-
ClawBench_Lite_V1.enc
20-
21-
# PyPI alias distributions are generated on demand by
22-
# ``scripts/build-aliases.py`` from a template — source of truth is the
23-
# root ``pyproject.toml`` and ``src/``, so these directories
24-
# are pure build output and never belong in git.
25-
packaging/aliases/
16+
.pytest_cache/
17+
run-logs/

CONTRIBUTING.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -126,7 +126,7 @@ Please use the [issue templates](https://github.com/reacher-z/ClawBench/issues/n
126126
Questions, task ideas, model submissions, or just want to chat about browser agents?
127127

128128
- **Discord:** [discord.gg/clawbench](https://discord.gg/clawbench) — English, agent-builder-friendly, `#contributors` channel
129-
- **微信群:** use the [QR code](./static/community/wechat_grp_422.jpg)
129+
- **微信群:** use the [QR code](./assets/community/wechat_grp_422.jpg)
130130
- **GitHub Discussions:** [github.com/reacher-z/ClawBench/discussions](https://github.com/reacher-z/ClawBench/discussions) — asynchronous, searchable
131131
- **Email:** open an issue; we respond there first
132132

README.md

Lines changed: 21 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -2,8 +2,8 @@
22

33
<a href="https://github.com/reacher-z/ClawBench">
44
<picture>
5-
<source media="(prefers-color-scheme: dark)" srcset="static/hero-dark.svg">
6-
<img alt="ClawBench" src="static/hero-light.svg" width="820">
5+
<source media="(prefers-color-scheme: dark)" srcset="assets/hero-dark.svg">
6+
<img alt="ClawBench" src="assets/hero-light.svg" width="820">
77
</picture>
88
</a>
99

@@ -62,7 +62,7 @@ git clone https://github.com/reacher-z/ClawBench.git && cd ClawBench && ./run.sh
6262

6363
**ClawBench is an open-source benchmark that evaluates AI browser agents on everyday online tasks — booking travel, ordering food, applying for jobs, managing email — across live websites. V1 lives in `test-cases/v1/` with 153 tasks across 144 websites; V2 lives in `test-cases/v2/` with 130 tasks. It measures end-to-end task success with a 5-layer recording pipeline and an agentic evaluator that compares each run against human references. Top score to date: 33.3%.**
6464

65-
<img src="static/clawbench_logo.png" alt="ClawBench logo" width="320">
65+
<img src="assets/clawbench_logo.png" alt="ClawBench logo" width="320">
6666

6767
We asked frontier AI agents to do what people do every day --<br/>
6868
order food, book travel, apply for jobs, write reviews, manage projects.<br/>
@@ -74,11 +74,11 @@ order food, book travel, apply for jobs, write reviews, manage projects.<br/>
7474

7575
**V1: 153** everyday tasks &nbsp;&middot;&nbsp; **V2: 130** tasks &nbsp;&middot;&nbsp; **144** live websites &nbsp;&middot;&nbsp; **15** life categories
7676

77-
<a href="README.zh-CN.md"><img src="static/icons/language.svg" width="16" height="16"> 中文</a>
77+
<a href="docs/README.zh-CN.md"><img src="assets/icons/language.svg" width="16" height="16"> 中文</a>
7878

7979
</div>
8080

81-
## <img src="static/icons/circle-question.svg" width="20" height="20"> What are you looking for?
81+
## <img src="assets/icons/circle-question.svg" width="20" height="20"> What are you looking for?
8282

8383
<table>
8484
<tr>
@@ -159,18 +159,18 @@ order food, book travel, apply for jobs, write reviews, manage projects.<br/>
159159
<br/>
160160

161161
<p align="center">
162-
<img src="static/icons/globe.svg" width="24" height="24">&nbsp;<b>Live Websites</b>
162+
<img src="assets/icons/globe.svg" width="24" height="24">&nbsp;<b>Live Websites</b>
163163
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
164-
<img src="static/icons/cube.svg" width="24" height="24">&nbsp;<b>Isolated Containers</b>
164+
<img src="assets/icons/cube.svg" width="24" height="24">&nbsp;<b>Isolated Containers</b>
165165
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
166-
<img src="static/icons/shield-halved.svg" width="24" height="24">&nbsp;<b>Request Interceptor</b>
166+
<img src="assets/icons/shield-halved.svg" width="24" height="24">&nbsp;<b>Request Interceptor</b>
167167
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
168-
<img src="static/icons/layer-group.svg" width="24" height="24">&nbsp;<b>Five-Layer Recording</b>
168+
<img src="assets/icons/layer-group.svg" width="24" height="24">&nbsp;<b>Five-Layer Recording</b>
169169
</p>
170170

171171
<br/>
172172

173-
## <img src="static/icons/layer-group.svg" width="20" height="20"> Datasets
173+
## <img src="assets/icons/layer-group.svg" width="20" height="20"> Datasets
174174

175175
ClawBench ships **three** Hugging Face datasets — task definitions plus full execution traces for V1 and V2. All open, downloadable in one command. The benchmark itself is also mirrored on **TIGER-Lab** for visibility.
176176

@@ -200,13 +200,13 @@ ClawBench ships **three** Hugging Face datasets — task definitions plus full e
200200

201201
<br/>
202202

203-
# <img src="static/icons/robot.svg" width="28" height="28"> LLM Quick Start
203+
# <img src="assets/icons/robot.svg" width="28" height="28"> LLM Quick Start
204204

205205
Point your coding agent (Claude Code, Cursor, Copilot, etc.) at [`AGENTS.md`](AGENTS.md) and prompt away.
206206

207207
<br/>
208208

209-
# <img src="static/icons/person.svg" width="28" height="28"> Human Quick Start
209+
# <img src="assets/icons/person.svg" width="28" height="28"> Human Quick Start
210210

211211
Install ClawBench from PyPI for normal use:
212212

@@ -348,7 +348,7 @@ This path gives you live-reload on ``src/``, ``src/clawbench/runtime/chrome-exte
348348

349349
<br/>
350350

351-
# <img src="static/icons/check-double.svg" width="28" height="28"> Reproduce the leaderboard
351+
# <img src="assets/icons/check-double.svg" width="28" height="28"> Reproduce the leaderboard
352352

353353
> **Our scores are stable**: two independent runs of the same model under the same judge (`deepseek/deepseek-v4-pro`, lenient rubric) reproduce Intercepted and Reward within ±2 pp on the V2 130-task corpus.
354354
@@ -386,7 +386,7 @@ For `deepseek-v4-flash:free × hermes × v2`, the published row is **Intercepted
386386

387387
<br/>
388388

389-
# <img src="static/icons/chart-bar.svg" width="28" height="28"> ClawBench-Lite
389+
# <img src="assets/icons/chart-bar.svg" width="28" height="28"> ClawBench-Lite
390390

391391
**New here? Run this first.** [`test-cases/v1-lite/`](test-cases/v1-lite/) is a **20-task curated subset** of the V1 153-task corpus, selected for household-name sites, real-world relevance, difficulty, and category diversity. It matches the 20-tasks-per-source convention of [browser-use/benchmark](https://github.com/browser-use/benchmark) and gives you a credible signal at a fraction of the full-benchmark cost.
392392

@@ -396,13 +396,13 @@ The Lite suite is a first-class task directory: run it with `--cases-suite v1-li
396396

397397
<br/>
398398

399-
# <img src="static/icons/play.svg" width="28" height="28"> Demos
399+
# <img src="assets/icons/play.svg" width="28" height="28"> Demos
400400

401401
Each ClawBench run produces a full MP4 session recording. See the [project page](https://claw-bench.com) for V1 task recordings.
402402

403403
<br/>
404404

405-
# <img src="static/icons/circle-question.svg" width="28" height="28"> Example Walkthrough
405+
# <img src="assets/icons/circle-question.svg" width="28" height="28"> Example Walkthrough
406406

407407
Curious what one task actually looks like, start to finish? Here's task **001** end to end.
408408

@@ -444,7 +444,7 @@ All four must hold for a **PASS**. Miss any one and it's a **FAIL** with evidenc
444444

445445
<br/>
446446

447-
# <img src="static/icons/chart-bar.svg" width="28" height="28"> Results
447+
# <img src="assets/icons/chart-bar.svg" width="28" height="28"> Results
448448

449449
<div align="center">
450450

@@ -610,7 +610,7 @@ ClawBench's niche: **live consumer websites, everyday tasks, end-to-end recordin
610610

611611
<br/>
612612

613-
# <img src="static/icons/terminal.svg" width="28" height="28"> CLI
613+
# <img src="assets/icons/terminal.svg" width="28" height="28"> CLI
614614

615615
```bash
616616
# Interactive TUI (recommended):
@@ -642,7 +642,7 @@ V1 tasks are in [`test-cases/v1/`](test-cases/v1/) (153 tasks). V2 tasks are in
642642

643643
<br/>
644644

645-
# <img src="static/icons/chart-bar.svg" width="28" height="28"> Evaluation
645+
# <img src="assets/icons/chart-bar.svg" width="28" height="28"> Evaluation
646646

647647
Evaluation is a **post-session** step -- first run agents to collect trajectories, then evaluate them against human reference runs.
648648

@@ -660,7 +660,7 @@ See [eval/README.md](eval/README.md) for the full evaluation guide and Claude Co
660660

661661
<br/>
662662

663-
# <img src="static/icons/circle-question.svg" width="28" height="28"> FAQ
663+
# <img src="assets/icons/circle-question.svg" width="28" height="28"> FAQ
664664

665665
<details>
666666
<summary><b>What data does each run produce?</b></summary>
@@ -753,7 +753,7 @@ Come hang out with researchers, builders, and contributors working on real-world
753753
<sub><b>English community</b><br/>Agent builders, researchers, contributors</sub>
754754
</td>
755755
<td align="center" width="33%">
756-
<a href="static/community/wechat_grp_422.jpg">
756+
<a href="assets/community/wechat_grp_422.jpg">
757757
<img src="https://img.shields.io/badge/%E5%BE%AE%E4%BF%A1%E7%BE%A4-%E5%8A%A0%E5%85%A5-07C160?style=for-the-badge&logo=wechat&logoColor=white" alt="微信群">
758758
</a>
759759
<br/>

0 commit comments

Comments
 (0)