Add caching integration tests#138
Closed
reyhankoyun wants to merge 12 commits intoaws:mainfrom
reyhankoyun:caching-tests-clean
Closed
Add caching integration tests#138reyhankoyun wants to merge 12 commits intoaws:mainfrom reyhankoyun:caching-tests-clean
reyhankoyun wants to merge 12 commits intoaws:mainfrom
reyhankoyun:caching-tests-clean
Conversation
- test_cache_hit_behavior: Verifies cache hits are faster than AWS calls - test_refresh_now_bypasses_cache: Confirms refreshNow=true bypasses cache - test_cache_after_secret_update: Tests stale cache behavior after secret updates - test_real_ttl_expiration_timing: Validates TTL expiration and cache refresh - test_ttl_zero_disables_caching: Ensures TTL=0 disables caching completely These tests cover all critical caching behaviors that cannot be unit tested, including timing-based assertions and AWS integration scenarios.
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #138 +/- ##
=======================================
Coverage 91.72% 91.72%
=======================================
Files 14 14
Lines 2418 2418
Branches 2418 2418
=======================================
Hits 2218 2218
Misses 150 150
Partials 50 50 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
simonmarty
added
safe-to-test
- Updated caching.rs to only include true integration tests: - test_cache_after_secret_update: Real AWS secret rotation + cache staleness - test_real_ttl_expiration_timing: Real time-based TTL with actual delays - Removed performance-focused tests (moved to future performance suite) - Removed parameter behavior tests (moved to future unit tests) - Fixed GitHub Actions security vulnerability: - Changed pull_request_target to only trigger on 'labeled' events - Eliminates race condition where unapproved code could execute with AWS credentials - Each commit now requires explicit human approval via safe-to-test label - Auto-removes label after use to prevent persistent approval Integration tests now focus on real AWS interactions and timing behavior that cannot be effectively mocked or measured in unit tests.
reyhankoyun
added
the
safe-to-test
simonmarty
requested changes
Oct 24, 2025
| - name: Remove safe-to-test label after use | ||
| if: github.event_name == 'pull_request_target' && contains(github.event.pull_request.labels.*.name, 'safe-to-test') | ||
| run: | | ||
| gh api repos/${{ github.repository }}/issues/${{ github.event.pull_request.number }}/labels/safe-to-test -X DELETE || true |
Contributor
There was a problem hiding this comment.
This is going to silence all errors that come out of the GitHub API
You can use something like
gh_status=$(gh api "repos/${{ github.repository }}/issues/${{ github.event.pull_request.number }}/labels/safe-to-test" -X DELETE | jq ".status" --raw-output)
case $gh_status in
200) echo "Label removed" ;;
404) echo "Label not found — ignoring" ;;
*) echo "unexpected HTTP $gh_status" && exit 1 ;;
to avoid that
| pull-requests: write | ||
|
|
||
| steps: | ||
| - name: Remove safe-to-test label after use |
Contributor
There was a problem hiding this comment.
One issue with doing this is that this will prevent re-driving integration workflows that fail due to causes outside of the code. Say there's an AWS outage while a workflow gets run, the label will get removed, then it will have to get re-added so that a new PR workflow can get triggered on the same commit, when in fact that relabeling should not be necessary from a safety standpoint, the commit has already been approved for integ tests by a person.
One way to get around this is to remove the safe-to-test label on the synchronize event like here
*Issue #, if available:* *Description of changes:* Extract the fix in 3f073a4 By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
Signed-off-by: Simon Marty <martysi@amazon.com>
simonmarty
added
safe-to-test
*Issue #, if available:* *Description of changes:* Extract the fix in 3f073a4 By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
Signed-off-by: Simon Marty <martysi@amazon.com>
simonmarty
added
safe-to-test
github-actions
bot
removed
the
safe-to-test
*Issue #, if available:* *Description of changes:* Extract the fix in 3f073a4 By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
Signed-off-by: Simon Marty <martysi@amazon.com>
Signed-off-by: Simon Marty <simon.marty@protonmail.com>
simonmarty
added
the
safe-to-test
github-actions
bot
removed
the
safe-to-test
simonmarty
added
the
safe-to-test
github-actions
bot
removed
the
safe-to-test
Signed-off-by: Simon Marty <martysi@amazon.com>
simonmarty
added
the
safe-to-test
github-actions
bot
removed
the
safe-to-test
lohdipak
reviewed
Oct 28, 2025
|
|
||
| #[tokio::test] | ||
| async fn test_real_ttl_expiration_timing() { | ||
| let secrets = TestSecrets::setup().await; |
There was a problem hiding this comment.
Instead of creating the secrets with pre defined name. can we update the setup to take secrets name prefix as input.
There was a problem hiding this comment.
we are creating multiple secrets in setup, do we need that for all the integration test? or for most of the test we just need single secrets
| let json3: serde_json::Value = serde_json::from_str(&response3).unwrap(); | ||
|
|
||
| // Should still get valid response after TTL expiry | ||
| assert!(json3["SecretString"].as_str().unwrap().contains("testuser")); |
There was a problem hiding this comment.
How are we validating that SMA fetch the data from aws request and cache refreshed.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Issue #, if available:
Description of changes:
Added caching integration tests:
test_cache_after_secret_update: Real AWS secret rotation + cache stalenesstest_real_ttl_expiration_timing: Real time-based TTL with actual delaysFixed GitHub Actions security vulnerability:
pull_request_targetto only trigger onlabeledeventssafe-to-testlabelBy submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.