Troubleshoot hanging rspec runs that caused job time outs - #549
Draft
jimtng wants to merge 3 commits into
Draft
Conversation
Signed-off-by: Jimmy Tanagra <jcode@tanagra.id.au>
Signed-off-by: Jimmy Tanagra <jcode@tanagra.id.au>
Signed-off-by: Jimmy Tanagra <jcode@tanagra.id.au>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Copilot assisted in this.
The branch now fails fast on bootstrap hangs and emits diagnostics before the GitHub runner kills the job.
Changes made
Bounded previously unbounded Karaf waits
Replaced infinite tracker waits with timeout-based waits in karaf.rb:385 and karaf.rb:397.
Added timeout helper methods that raise with context instead of blocking forever in karaf.rb:583.
Startup wait now fails with context instead of silently continuing
wait_for_start now uses an explicit timeout and label in karaf.rb:534.
Core wait method now raises on timeout in karaf.rb:560.
Added bundle state summary to timeout errors in karaf.rb:594.
CI RSpec step now gives heartbeat and thread dump on long hang
Wrapped RSpec run with heartbeat output every 30s in ci.yml:165.
If elapsed reaches 330s, sends SIGQUIT (thread dump), then TERM/KILL, exits 124 in ci.yml:178.
Added karaf.log to failure artifact upload in ci.yml:197.