Skip to content

Add benchmark workflow#153

Merged
simonmarty merged 20 commits intoaws:mainfrom
reyhankoyun:benchmark-workflow-fix
Feb 3, 2026
Merged

Add benchmark workflow#153
simonmarty merged 20 commits intoaws:mainfrom
reyhankoyun:benchmark-workflow-fix

Conversation

@reyhankoyun
Copy link
Copy Markdown
Contributor

@reyhankoyun reyhankoyun commented Dec 10, 2025

Issue #, if available:

Description of changes:
This PR builds off of PR #122 and creates a benchmark workflow similar to integration tests workflow that runs the benchmarks added in PR #122

Add two basic benchmarks

  • One that performs cache hit retrievals
  • One that intentionally yields cache misses on every retrieval

Using criterion (used by the AWS SDK for Rust).

This hopefully will serve to catch performance regressions in future code changes and open the door to profiling.

Successful run from fork: https://github.com/reyhankoyun/aws-secretsmanager-agent/actions/runs/20082405961/job/57612354790

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

@reyhankoyun reyhankoyun requested a review from a team as a code owner December 10, 2025 00:24
@reyhankoyun reyhankoyun added the safe-to-test Maintainer approval to run integration tests for external contributor PRs. label Dec 10, 2025
@github-actions github-actions bot removed the safe-to-test Maintainer approval to run integration tests for external contributor PRs. label Dec 10, 2025
@reyhankoyun reyhankoyun changed the title Fix YAML indentation in benchmarks workflow Add benchmark workflow Dec 10, 2025
@codecov
Copy link
Copy Markdown

codecov bot commented Dec 10, 2025

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 92.14%. Comparing base (7c6a7bc) to head (bb9e1f6).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #153      +/-   ##
==========================================
+ Coverage   91.88%   92.14%   +0.25%     
==========================================
  Files          14       14              
  Lines        2404     2405       +1     
  Branches     2404     2405       +1     
==========================================
+ Hits         2209     2216       +7     
+ Misses        147      143       -4     
+ Partials       48       46       -2     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@reyhankoyun reyhankoyun added the safe-to-test Maintainer approval to run integration tests for external contributor PRs. label Dec 16, 2025
@github-actions github-actions bot removed the safe-to-test Maintainer approval to run integration tests for external contributor PRs. label Dec 16, 2025
@reyhankoyun reyhankoyun added the safe-to-test Maintainer approval to run integration tests for external contributor PRs. label Dec 16, 2025
@github-actions github-actions bot removed the safe-to-test Maintainer approval to run integration tests for external contributor PRs. label Dec 16, 2025
@reyhankoyun reyhankoyun added the safe-to-test Maintainer approval to run integration tests for external contributor PRs. label Dec 16, 2025
@github-actions github-actions bot removed the safe-to-test Maintainer approval to run integration tests for external contributor PRs. label Dec 16, 2025
@reyhankoyun reyhankoyun added the safe-to-test Maintainer approval to run integration tests for external contributor PRs. label Dec 16, 2025
@github-actions github-actions bot removed the safe-to-test Maintainer approval to run integration tests for external contributor PRs. label Dec 16, 2025
Copy link
Copy Markdown
Contributor

@simonmarty simonmarty left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks like the upload is not succeeding with

Warning: No files were found with the provided path: aws_secretsmanager_caching/target/criterion/. No artifacts will be uploaded.

@reyhankoyun reyhankoyun added the safe-to-test Maintainer approval to run integration tests for external contributor PRs. label Dec 18, 2025
@github-actions github-actions bot removed the safe-to-test Maintainer approval to run integration tests for external contributor PRs. label Dec 18, 2025
@reyhankoyun reyhankoyun added the safe-to-test Maintainer approval to run integration tests for external contributor PRs. label Dec 18, 2025
@github-actions github-actions bot removed the safe-to-test Maintainer approval to run integration tests for external contributor PRs. label Dec 18, 2025
Copy link
Copy Markdown
Member

@ThirdEyeSqueegee ThirdEyeSqueegee left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The Criterion book has a section about benchmarking async functions. Did we take this into consideration when writing these tests? Not sure how it plays with Tokio.

@reyhankoyun
Copy link
Copy Markdown
Contributor Author

The Criterion book has a section about benchmarking async functions. Did we take this into consideration when writing these tests? Not sure how it plays with Tokio.

Yes, we're already implementing Criterion's async benchmarking pattern for Tokio.

- Download previous benchmark results before running benchmarks
- Use branch-based artifact naming for baseline comparison
- Add continue-on-error for first runs without baseline
- Enables automatic performance regression detection across runs
@reyhankoyun reyhankoyun force-pushed the benchmark-workflow-fix branch from 7438445 to 969e781 Compare January 29, 2026 21:33
@reyhankoyun reyhankoyun added the safe-to-test Maintainer approval to run integration tests for external contributor PRs. label Jan 29, 2026
@github-actions github-actions bot removed the safe-to-test Maintainer approval to run integration tests for external contributor PRs. label Jan 29, 2026
@reyhankoyun reyhankoyun added the safe-to-test Maintainer approval to run integration tests for external contributor PRs. label Jan 29, 2026
@github-actions github-actions bot removed the safe-to-test Maintainer approval to run integration tests for external contributor PRs. label Jan 29, 2026
@reyhankoyun reyhankoyun added the safe-to-test Maintainer approval to run integration tests for external contributor PRs. label Jan 29, 2026
@github-actions github-actions bot removed the safe-to-test Maintainer approval to run integration tests for external contributor PRs. label Jan 29, 2026
@reyhankoyun reyhankoyun force-pushed the benchmark-workflow-fix branch from 94614bb to e6ea6f4 Compare February 3, 2026 19:22
@reyhankoyun reyhankoyun added the safe-to-test Maintainer approval to run integration tests for external contributor PRs. label Feb 3, 2026
@github-actions github-actions bot removed the safe-to-test Maintainer approval to run integration tests for external contributor PRs. label Feb 3, 2026
@reyhankoyun reyhankoyun added the safe-to-test Maintainer approval to run integration tests for external contributor PRs. label Feb 3, 2026
@github-actions github-actions bot removed the safe-to-test Maintainer approval to run integration tests for external contributor PRs. label Feb 3, 2026
@simonmarty simonmarty merged commit d37e917 into aws:main Feb 3, 2026
12 checks passed
@reyhankoyun reyhankoyun mentioned this pull request Feb 4, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants