Add ordered indexes for job fetches - #1
Draft
bcomnes wants to merge 1 commit into
Draft
Conversation
Coverage Report for CI Build 29880243419Coverage decreased (-0.07%) to 97.799%Details
Uncovered ChangesNo uncovered changes found. Coverage RegressionsNo coverage regressions found. Coverage Stats💛 - Coveralls |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
job_i10 (name, priority DESC, created_on, id)for priority-ordered fetchesjob_i11 (name, created_on, id)for FIFO fetchesstate < 'active' AND NOT blockedpredicatejob_i5 (name, start_after)for queues dominated by future-scheduled jobsWhy
The existing fetch index can filter by queue and schedule time, but it cannot satisfy either fetch ordering. PostgreSQL may therefore scan and sort the eligible working set before applying
LIMIT, and a plan/statistics change can make that work unexpectedly expensive.The ordered indexes let PostgreSQL walk candidates in the required order and stop as soon as the batch is full. This is independent of any query-shape changes.
Migration impact
CREATE INDEX CONCURRENTLYthrough BAM forjob_commonand every existing partition.create_queuefunction gives future partitions the indexes at creation time.jobtable.Validation
npx vitest run: 65 files, 844 tests passednpx eslint .npm run tscnpm run buildnpm run gen:manifest:checkjob_common_i10job_common_i11