Skip to content

Commit b6f1af5

Browse files
authored
Merge pull request #283 from TIGER-AI-Lab/docs/readme-slim
docs: slim the README, fix broken links and corpus counts
2 parents f0e0584 + b39797b commit b6f1af5

13 files changed

Lines changed: 1232 additions & 995 deletions

AGENTS.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@ This file is for coding agents (Claude Code, Cursor, Copilot, etc.) to understan
44

55
## What This Is
66

7-
ClawBench is a benchmarking framework for evaluating AI web agents on real-world online tasks. V1 lives in `test-cases/v1/` with 153 tasks spanning 144 live websites and 15 life categories; V2 lives in `test-cases/v2/` with 130 tasks; Lite lives in `test-cases/v1-lite/` with 20 link-backed V1 tasks. Each task runs in an isolated Docker container with a browser, a recording Chrome extension, and an AI agent harness (`openclaw`, `opencode`, `claude-code`, `claude-code-chrome-extension`, `codex`, `browser-use`, `claw-code`, `hermes`, `pi`, or `webbrain`, selectable via `--harness`). The framework captures five layers of data: session replay (MP4), action screenshots, HTTP traffic, browser actions, and agent messages.
7+
ClawBench is a benchmarking framework for evaluating AI web agents on real-world online tasks. V1 lives in `test-cases/v1/` with 152 tasks spanning 143 live websites and 15 life categories; V2 lives in `test-cases/v2/` with 129 tasks; Lite lives in `test-cases/v1-lite/` with 20 link-backed V1 tasks. Each task runs in an isolated Docker container with a browser, a recording Chrome extension, and an AI agent harness (`openclaw`, `opencode`, `claude-code`, `claude-code-chrome-extension`, `codex`, `browser-use`, `claw-code`, `hermes`, or `pi`, or `webbrain`, selectable via `--harness`). The framework captures five layers of data: session replay (MP4), action screenshots, HTTP traffic, browser actions, and agent messages.
88

99
## Project Structure
1010

@@ -34,8 +34,8 @@ ClawBench/
3434
model.schema.json # JSON schema for model entries
3535
test-cases/
3636
task.schema.json # JSON schema for task.json
37-
v1/ # V1: 153 task directories
38-
v2/ # V2: 130 task directories
37+
v1/ # V1: 152 task directories
38+
v2/ # V2: 129 task directories
3939
v1-lite/ # Lite: 20 V1 task directories with linked task files
4040
eval/
4141
README.md # Evaluation guide + Claude Code prompt template

CONTRIBUTING.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -32,7 +32,7 @@ If no `good first issue` is currently open, just open one with your idea and we'
3232

3333
## Adding a new test case
3434

35-
ClawBench currently has two full task corpora: V1 lives in `test-cases/v1/` with 153 tasks, and V2 lives in `test-cases/v2/` with 130 tasks. Both use `test-cases/task.schema.json`. Unless a maintainer asks for a V2-only contribution, add new tasks to V1.
35+
ClawBench currently has two full task corpora: V1 lives in `test-cases/v1/` with 152 tasks, and V2 lives in `test-cases/v2/` with 129 tasks. Both use `test-cases/task.schema.json`. Unless a maintainer asks for a V2-only contribution, add new tasks to V1.
3636

3737
1. **Pick a task ID** — find the next available number by checking existing directories in the target corpus (`test-cases/v1/` for V1, or `test-cases/v2/` for V2).
3838

@@ -125,9 +125,9 @@ Please use the [issue templates](https://github.com/reacher-z/ClawBench/issues/n
125125

126126
Questions, task ideas, model submissions, or just want to chat about browser agents?
127127

128-
- **Discord:** [discord.gg/clawbench](https://discord.gg/clawbench)English, agent-builder-friendly, `#contributors` channel
128+
- **GitHub issues:** [open one](https://github.com/TIGER-AI-Lab/ClawBench/issues/new/choose)fastest route to a maintainer
129129
- **微信群:** use the [QR code](./assets/community/wechat_grp_422.jpg)
130-
- **GitHub Discussions:** [github.com/reacher-z/ClawBench/discussions](https://github.com/reacher-z/ClawBench/discussions) — asynchronous, searchable
130+
- **Hugging Face:** [dataset discussions](https://huggingface.co/datasets/NAIL-Group/ClawBench/discussions) — asynchronous, searchable, next to the data
131131
- **Email:** open an issue; we respond there first
132132

133133
## Code of conduct

README.md

Lines changed: 375 additions & 493 deletions
Large diffs are not rendered by default.

assets/hero-dark.svg

Lines changed: 1 addition & 1 deletion
Loading

assets/hero-light.svg

Lines changed: 1 addition & 1 deletion
Loading

assets/icons/check-double.svg

Lines changed: 1 addition & 0 deletions
Loading

assets/social-card.png

144 KB
Loading

0 commit comments

Comments
 (0)