Skip to content

Add mujoco/model to benchmarks #87

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
ft-lab wants to merge 9 commits into
base: main
Choose a base branch
from
Open

Add mujoco/model to benchmarks #87

Show file tree
Hide file tree
Changes from 4 commits
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
369d5fc
Updated benchmarks.
ft-lab May 11, 2026
b842805
Merge branch 'main' into benchmarks
ft-lab May 11, 2026
f93384c
Benchmark: Updated benchmark.yml
ft-lab May 11, 2026
20882da
Updated benchmarks.md
ft-lab May 11, 2026
9596bbd
Updated manage_anotations.py
ft-lab May 12, 2026
92a9f8b
Updated README.md
ft-lab May 12, 2026
1665110
Updated tools/benchmark_and_report.py
ft-lab May 12, 2026
78bd299
Updated tools/benchmark_and_report.py
ft-lab May 12, 2026
e409fe8
Updated benchmark_and_report.py
ft-lab May 14, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 22 additions & 2 deletions .github/workflows/benchmark.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,8 +23,28 @@ jobs:
- name: Bootstrap
uses: ./.github/actions/bootstrap

- name: Run benchmarks
run: uv run --group dev poe benchmark
- name: Run benchmarks for all repositories
run: |
# Define repositories as arrays
declare -a urls=(
"https://github.com/google-deepmind/mujoco_menagerie"
"https://github.com/google-deepmind/mujoco"
)

# Build arguments array with --mjcf-repository-path for each URL
args=()
for url in "${urls[@]}"; do
args+=(--mjcf-repository-path "$url")
done

# Run benchmarks for all repositories in a single command
echo "============================================"
echo "Running benchmarks for all repositories:"
for url in "${urls[@]}"; do
echo " - $url"
done
echo "============================================"
uv run --group dev poe benchmark "${args[@]}"
Comment thread
ft-lab marked this conversation as resolved.
Outdated

- name: Upload benchmark results
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
Expand Down
239 changes: 148 additions & 91 deletions benchmarks.md
View file
Open in desktop

Large diffs are not rendered by default.

14 changes: 7 additions & 7 deletions pyproject.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -64,16 +64,16 @@ help = "Show coverage report in terminal"
cmd = "coverage report -m"

[tool.poe.tasks.benchmark]
help = "Run MuJoCo Menagerie benchmark"
cmd = "python tools/benchmark_menagerie.py"
help = "Run MuJoCo benchmark"
cmd = "python tools/benchmark_and_report.py"

[tool.poe.tasks.benchmark-publish]
help = "Run MuJoCo Menagerie benchmark and output the markdown report so it overwrites the committed report"
cmd = "python tools/benchmark_menagerie.py --report-format md --report-output-dir ."
help = "Run MuJoCo benchmark and output the markdown report so it overwrites the committed report"
cmd = "python tools/benchmark_and_report.py --report-format md --report-output-dir ."

[tool.poe.tasks.benchmark-csv]
help = "Run MuJoCo Menagerie benchmark and generate CSV report only"
cmd = "python tools/benchmark_menagerie.py --report-format csv"
help = "Run MuJoCo benchmark and generate CSV report only"
cmd = "python tools/benchmark_and_report.py --report-format csv"

[tool.poe.tasks.annotations]
help = "Show annotation summary"
Expand All @@ -84,7 +84,7 @@ help = "Validate annotation file"
cmd = "python tools/manage_annotations.py --validate"

[tool.poe.tasks.annotations-update]
help = "Update annotation file with new assets from Menagerie"
help = "Update annotation file with new assets"
cmd = "python tools/manage_annotations.py --update"

[tool.poe.tasks.lint]
Expand Down
Loading
Loading