Extract archive push into ArchiveIndexBranch background job#13472
Open
Turbo87 wants to merge 8 commits intorust-lang:mainfrom
Open
Extract archive push into ArchiveIndexBranch background job#13472Turbo87 wants to merge 8 commits intorust-lang:mainfrom
ArchiveIndexBranch background job#13472Turbo87 wants to merge 8 commits intorust-lang:mainfrom
Conversation
Turbo87
force-pushed
the
archive-index-branch-job
branch
from
f481b6d to
a56c046
Compare
This comment has been minimized.
This comment has been minimized.
Reading the env var once at startup instead of on every job run, and making it overridable per test via `with_config(|c| ...)`. Named `index_archive_url` to distinguish it from unrelated archives (e.g. version-downloads).
`run_pending_background_jobs()` panics when any job fails, which makes it impossible for tests to deliberately exercise a job's failure path. Factor out a `try_*` variant that returns the `check_for_failed_jobs` result so callers can observe failures explicitly.
Mirrors a snapshot branch from the crate index to the `index_archive_url` configured on `config::Server`, or no-ops if no URL is configured. Runs on the `repository` queue with payload deduplication.
Lets operators manually enqueue `ArchiveIndexBranch` jobs for a given snapshot branch name, e.g. to re-run a push that was lost because the archive remote was down.
Drop the inline push to `index_archive_url` and delegate it to the dedicated `ArchiveIndexBranch` background job. The blocking squash+push now returns the snapshot branch name; the outer async code enqueues the follow-up job, warning but not failing on connection or enqueue errors.
Turbo87
force-pushed
the
archive-index-branch-job
branch
from
a56c046 to
6c89397
Compare
Collaborator
|
This PR was rebased onto a different main commit. Here's a range-diff highlighting what actually changed. Rebasing is a normal part of keeping PRs up to date, so no action is needed—this note is just to help reviewers. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Extract the archive push from
SquashIndexinto a newArchiveIndexBranchbackground job. The job readsGIT_ARCHIVE_REPO_URLfromconfig::Server(loaded once at startup) and is a no-op if no URL is configured.SquashIndexenqueues it after a successful squash-and-push, warning but not failing if the enqueue itself fails. The job is also manually enqueuable viacrates-admin enqueue-job archive_index_branch <branch>.This change ensures that a failing archival will not unintentionally retrigger another index squash. It also allows us in the future to move the
ArchiveIndexBranchjob to a low-priority job on the default background worker queue, which unlocks the regular index repository earlier, allowing the git index sync jobs to continue earlier too.Related