feat(jobs): add state, limit, ordering and cursor filters to findJobs() - #889
Draft
kibertoad wants to merge 1 commit into
Draft
feat(jobs): add state, limit, ordering and cursor filters to findJobs()#889kibertoad wants to merge 1 commit into
kibertoad wants to merge 1 commit into
Conversation
findJobs() has no bound and no order. Every result set is the queue's entire
retained history for the given filters, in whatever order the planner
produces, which makes it unusable for the admin and inspection work it is
meant for: a key with a year of history returns a year of history, and there
is no way to ask for the last twenty.
Adds four options that turn it into a paging read:
- states: filter on an explicit list of job states. queued already covers
created + retry; states covers any subset, including terminal ones.
Combining the two is rejected rather than silently resolved.
- limit: bound the result.
- orderBy (createdOn | startAfter) and direction: define the order. Both
columns are immutable for the life of a job, so a job that changes state
mid-pagination keeps its place.
- cursor: keyset pagination on (order column, id). The anchor's sort value is
read back from the row the cursor names and the row comparison walks
strictly past it, so a row inserted or deleted between pages cannot shift
the window and page N does not cost more than page 1. id breaks ties so the
order is total.
Ordering stays opt-in: a call that supplies none of limit, orderBy, direction
or cursor gets the statement findJobs has always issued, unordered and with
the plan it has today.
Also adds getJobByKey(name, key, options), the singleton-key counterpart of
getJobById(). A key names a series rather than a row, so it answers with the
most recent job, and { queued: true } narrows it to what the key currently has
outstanding.
findJobs() now applies the CockroachDB integer normalization fetch() and
getJobById() already perform, so a job read through it has the same shape on
every backend.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
findJobs()has no bound and no order. Every call returns the queue's entire retained history for the given filters, in whatever order the planner produces. That makes it awkward for the admin and inspection work it exists for:findJobs('email-send', { key: 'user-123' })on a key that has been busy for a year returns a year of jobs. There is no way to ask for the last twenty.queued, which only expressescreated+retry. "Show me the failed ones" is a full table read.This adds four options, plus a companion read.
statesqueuedis the special case['created', 'retry']. Supplying both is rejected rather than silently resolved in favour of one:{ queued: true, states: ['completed'] }has no reading that a caller would predict.limit,orderBy,directionorderByiscreatedOn(default) orstartAfter. Both are immutable for the life of a job, which is what makes them safe to sort on: a job that changes state mid-pagination keeps its place in the ordering instead of jumping to another page.cursorKeyset pagination on
(order column, id), notOFFSET:The anchor's sort value is read back from the row the cursor names and the row comparison walks strictly past it. A job inserted or deleted between pages cannot shift the window into repeating or skipping a row, and page N does not cost more than page 1.
idis the tiebreaker so the order is total: without it, two jobs sharing acreated_oncould sort either way between calls and land on both sides of a page boundary.getJobByKey(name, key, options)The singleton-key counterpart of
getJobById(). A key identifies a series rather than a row, so it answers with the most recent job:Compatibility
Ordering is opt-in. A call that supplies none of
limit,orderBy,direction, orcursorproduces the statementfindJobs()has always issued, unordered, with the plan it has today. No existing call changes shape or cost.One drive-by fix:
findJobs()now applies the CockroachDB integer normalizationfetch()andgetJobById()already perform, so a job read through it has the same shape on every backend. Without itgetJobByKey()would returnretryLimitas a string wheregetJobById()returns a number.Tests
test/findJobsFiltersTest.ts: state filtering including the rejected combinations, bounded results, both orderings, forward and backward paging, a cursor naming a deleted row's successor (the case an offset pager gets wrong), a cursor naming nothing, state filter combined with paging, and the unordered path staying unordered. PlusgetJobByKeynewest-first, thequeuednarrowing, key isolation, and the null cases.