Skip to content

Add CodSpeed continuous performance testing - #2493

Closed
codspeed-hq[bot] wants to merge 2 commits into
masterfrom
codspeed-wizard-1776465467978
Closed

Add CodSpeed continuous performance testing#2493
codspeed-hq[bot] wants to merge 2 commits into
masterfrom
codspeed-wizard-1776465467978

Conversation

@codspeed-hq

@codspeed-hq codspeed-hq Bot commented Apr 17, 2026

Copy link
Copy Markdown

Summary

This PR integrates CodSpeed for continuous performance testing on the moonrepo/moon repository. CodSpeed provides accurate, reproducible performance measurements in CI, enabling detection of performance regressions on every pull request.

Changes

Dependency update -- Replaced criterion2 with codspeed-criterion-compat (v4.5.0) in:

  • Cargo.toml (workspace dependency)
  • crates/affected/Cargo.toml
  • crates/workspace/Cargo.toml

The compatibility layer is a drop-in replacement for criterion, so no changes were needed in the benchmark source files (crates/affected/benches/affected_tracker.rs and crates/workspace/benches/workspace_graphs.rs). When not running under cargo codspeed, it behaves identically to standard criterion, so cargo bench continues to work as before.

CI workflow -- Added .github/workflows/codspeed.yml that:

  • Triggers on pushes to master, pull requests (scoped to Rust-relevant paths), and workflow_dispatch (for CodSpeed backtest analysis)
  • Uses OIDC authentication (id-token: write)
  • Builds and runs benchmarks with cargo codspeed in simulation mode
  • Uses CodSpeedHQ/action@v4

README -- Added the CodSpeed performance badge.

Next steps

  1. Install the CodSpeed GitHub App on the moonrepo organization if not already done: https://github.com/apps/codspeed
  2. Activate the repository on CodSpeed at https://codspeed.io
  3. Once merged, CodSpeed will automatically run a backtest via workflow_dispatch to generate baseline data
  4. Performance reports will appear as PR comments on subsequent pull requests

Existing benchmarks covered

  • moon_affected -- Tracks affected projects/tasks (sync and async) at scales of 10, 100, and 1000 projects
  • moon_workspace -- Workspace graph building (sync and async) at scales of 100 and 1000 projects

@codspeed-hq
codspeed-hq Bot force-pushed the codspeed-wizard-1776465467978 branch from 2628625 to 71c76ab Compare April 17, 2026 23:18
@codspeed-hq

codspeed-hq Bot commented Apr 17, 2026

Copy link
Copy Markdown
Author

CodSpeed setup failed

The CodSpeed integration is fully implemented and committed locally (branch: codspeed-wizard-1776465467978), but could not be successfully deployed. PR #2493 exists but its CI workflow fails with OOM during the benchmark build step because the workspace's release profile (inherited by bench) uses LTO and codegen-units=1. The fix — overriding these via CARGO_PROFILE_BENCH_LTO=false and CARGO_PROFILE_BENCH_CODEGEN_UNITS=16 environment variables — is committed locally (commit cbbb464) but could not be pushed: the authentication token expired and the PR submission tool reached its maximum of 3 attempts. The workflow file and all code changes are correct and ready to be pushed once authentication is restored.

@milesj milesj closed this Apr 18, 2026
@milesj
milesj deleted the codspeed-wizard-1776465467978 branch April 18, 2026 20:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

1 participant