manager.py: Fix shutdown signaling and tight retry loop in download job manager#47
Open
MateoGonzalezLourido wants to merge 2 commits into
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Three related fixes to download_job_manager and _do_chunk_signing:
When download_job_manager receives a TerminateWorkerTask, it now sets self.running = False before breaking. This signals _do_chunk_signing to exit its loop cleanly instead of continuing to process chunks for a download that's already terminating.
Previously, sign_pipe.recv() could block indefinitely even after self.running was set to False. Replaced with a poll(1.0) loop that checks self.running on each iteration, so the thread exits cleanly within ≤1 second of receiving the terminate signal.
The poll(1.0) loop is not busy-wait — it blocks at the OS level until data arrives or the timeout expires. In the normal case the signing response arrives in milliseconds and the loop never iterates more than once. The only case where multiple iterations occur is during shutdown, where ≤1 second of latency is acceptable. A shorter timeout would only add more iterations with no benefit.
3. Set no_signed_chunks = True in the exception handler
When dl_worker_queue.put() raises an exception, the outer loop now waits before retrying instead of spinning in a tight retry loop.