Skip to content
This repository was archived by the owner on Jul 10, 2026. It is now read-only.

fix: benchmark comparison workflow suffix naming - #110

Closed
0xShaito wants to merge 1 commit into
devfrom
fix/benchmark-comparison-workflow
Closed

fix: benchmark comparison workflow suffix naming#110
0xShaito wants to merge 1 commit into
devfrom
fix/benchmark-comparison-workflow

Conversation

@0xShaito

@0xShaito 0xShaito commented Feb 1, 2026

Copy link
Copy Markdown
Member

Summary

  • Fix benchmark comparison workflow showing Base: 0 by correcting suffix naming mismatch
  • Add explicit benchmark execution step in PR workflow
  • Use only_report: true flag to skip external action's unreliable benchmark execution

Root Cause

The comparison was failing because:

  1. update-baseline.yml created files with _latest suffix
  2. PR workflow expected to download baseline and compare with _new suffix
  3. External action's implicit benchmark execution via npx aztec-benchmark was unreliable
  4. No _new files were generated → comparison returned empty/zero results

Changes

.github/actions/benchmark/action.yml:

  • Add explicit Run benchmarks step: yarn aztec-benchmark --suffix _latest
  • Add reports_dir: ${{ inputs.bench-dir }}
  • Change base_suffix from '_latest' to '_baseline'
  • Change current_suffix from '_new' to '_latest'
  • Add only_report: true to use external action only for comparison, not execution
  • Update file cleanup/upload paths to use _baseline suffix

.github/workflows/update-baseline.yml:

  • Change benchmark suffix from _latest to _baseline
  • Update file listing and upload paths to use _baseline suffix

New Flow

update-baseline.yml (on push to dev/main):

  1. Runs yarn aztec-benchmark --suffix _baseline
  2. Uploads *_baseline.benchmark.json as artifact

benchmark/action.yml (on PR):

  1. Downloads baseline artifact (*_baseline.benchmark.json)
  2. Runs yarn aztec-benchmark --suffix _latest (generates current)
  3. External action compares _baseline vs _latest (only_report: true)
  4. Posts comparison, renames files to _baseline, uploads for PR branch

Test plan

  • After merge, update-baseline.yml workflow runs and creates baseline with _baseline suffix
  • Future PRs correctly download baseline and generate comparison reports with proper values

🤖 Generated with Claude Code

- Add explicit benchmark run step using yarn aztec-benchmark --suffix _latest
- Add reports_dir parameter to external action
- Change base_suffix from '_latest' to '_baseline'
- Change current_suffix from '_new' to '_latest'
- Add only_report: true to skip external action's benchmark execution
- Update file cleanup and upload paths to use _baseline suffix

This fixes the comparison showing Base: 0 because:
1. update-baseline.yml now creates files with _baseline suffix
2. PR workflow runs benchmarks with _latest suffix
3. External action only compares (only_report: true), doesn't run benchmarks
4. Naming is now consistent: baseline=_baseline, current=_latest

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
@github-actions

github-actions Bot commented Feb 1, 2026

Copy link
Copy Markdown

Benchmark Comparison

CPU Cores RAM Arch
N/A N/A N/A N/A

Contract: counter

Function Gates DA Gas L2 Gas Proving Time (ms)
Status Base PR Diff Base PR Diff Base PR Diff Base PR Diff
🆕 increment 0 472,292 +Inf% 0 2,253 +Inf% 0 28,872 +Inf% N/A N/A

@0xShaito

0xShaito commented Feb 1, 2026

Copy link
Copy Markdown
Member Author

Closing - reverting to original naming on test/benchmark instead.

@0xShaito 0xShaito closed this Feb 1, 2026
@0xShaito
0xShaito deleted the fix/benchmark-comparison-workflow branch February 1, 2026 02:58
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Development

Successfully merging this pull request may close these issues.

1 participant