feat: add generic ProgressType parameter to Job and Worker - #4529
feat: add generic ProgressType parameter to Job and Worker#4529skrukwa wants to merge 2 commits into
Conversation
There was a problem hiding this comment.
Pull request overview
This PR extends BullMQ’s TypeScript surface to support a user-defined, type-safe job progress shape by introducing an optional ProgressType generic (constrained to JobProgress and defaulting to JobProgress) across MinimalJob, Job, Processor, Worker, and WorkerListener, and documents/validates the new typing via docs and tests.
Changes:
- Add
ProgressType extends JobProgress = JobProgresstoMinimalJobandJob, typingprogressandupdateProgress()accordingly. - Thread
ProgressTypethroughWorkerandWorkerListenersoworker.on('progress', ...)and processorjob.updateProgress(...)become type-safe. - Add docs + a worker test demonstrating typed progress usage.
Reviewed changes
Copilot reviewed 6 out of 6 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| tests/worker.test.ts | Adds a test covering typed progress generics on Worker and the progress event. |
| src/types/processor.ts | Adds a 4th generic parameter to Processor so processor functions get typed job.updateProgress(). |
| src/interfaces/minimal-job.ts | Adds ProgressType generic and types progress/updateProgress() on the minimal job contract. |
| src/classes/worker.ts | Threads ProgressType through Worker and WorkerListener event typing and processor typing. |
| src/classes/job.ts | Adds ProgressType generic and types progress field + updateProgress(). |
| docs/gitbook/guide/workers/README.md | Documents how to use the new ProgressType generic with Worker. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
36d5556 to
b1f4e62
Compare
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 6 out of 6 changed files in this pull request and generated no new comments.
Suppressed comments (3)
src/classes/job.ts:70
Job.progressis typed asProgressType, but it is initialized to the numeric default0. If a user supplies a non-numericProgressType(e.g. an object shape), TypeScript will allow property access onjob.progressthat will fail at runtime until the firstupdateProgress()call. Consider reflecting the persisted default by including0in the type instead of casting.
progress: ProgressType = 0 as ProgressType;
src/classes/worker.ts:405
createSandboxis only used for sandboxed processors, which intentionally keepJobProgressbecause progress crosses a process boundary. Having the sandboxed-processor overload acceptProcessor<..., ProgressType>is misleading for subclasses and suggests typed progress is preserved in sandbox mode. Consider using the defaultProcessor<..., JobProgress>form here (omit the ProgressType parameter).
| Processor<DataType, ResultType, NameType, ProgressType>,
src/interfaces/minimal-job.ts:80
MinimalJob.progressis declared asProgressType, but BullMQ initializes progress to0. For custom progress shapes this makes the public type unsound (progress can be a number before the first update). Align the interface with runtime by allowing the default0value in the type.
progress: ProgressType;
The progress type was hardcoded to JobProgress (string | boolean | number | object), so there was no way to get type-safe progress when calling updateProgress() or when listening to the progress event. Add an optional ProgressType generic parameter, constrained to JobProgress and defaulting to JobProgress, to MinimalJob, Job, Processor, Worker and WorkerListener. Existing code that passes fewer generics keeps compiling unchanged. Because ProgressType is constrained to JobProgress, Job<..., P> stays assignable to the internal MinimalJob helpers, which never read or write progress, so the generic does not need to be threaded through them. This also keeps the persistence contract intact, since progress is ultimately serialized before being stored in the backend. On Worker the parameter is added after the backend generic B so that the existing public signature is not broken. Sandboxed processors keep JobProgress by design, as progress crosses a process boundary where the static type is lost. Queue side getters such as getJob and Job.fromId also keep the default. Closes taskforcesh#3721 Co-authored-by: claygeo <claygeo6@gmail.com>
b1f4e62 to
30c4eea
Compare
The progress type was hardcoded to JobProgress
(string | boolean | number | object), so there was no way to get type-safe progress when calling updateProgress() or when listening to the progress event.
Add an optional ProgressType generic parameter, constrained to JobProgress and defaulting to JobProgress, to MinimalJob, Job, Processor, Worker and WorkerListener. Existing code that passes fewer generics keeps compiling unchanged.
Because ProgressType is constrained to JobProgress, Job<..., P> stays assignable to the internal MinimalJob helpers, which never read or write progress, so the generic does not need to be threaded through them. This also keeps the persistence contract intact, since progress is ultimately serialized before being stored in the backend.
On Worker the parameter is added after the backend generic B so that the existing public signature is not broken.
Sandboxed processors keep JobProgress by design, as progress crosses a process boundary where the static type is lost. Queue side getters such as getJob and Job.fromId also keep the default.
Closes #3721
Port Impact Checklist
TypeScript types only — no runtime or Lua changes, nothing to port.
Why
JobProgressis hardcoded asstring | boolean | number | object, so progress can't be type-safe:Closes #3721 (also discussed in #3184).
Continues #3909 by @claygeo, closed when the head repo was deleted. Rebased onto current
masterand reworked for the v6 backend generic. @claygeo credited as co-author.How
Add an optional
ProgressTypegeneric toMinimalJob,Job,Processor,WorkerandWorkerListener, constrainedextends JobProgressand defaulting toJobProgress. Existing code with fewer generics compiles unchanged.Two decisions worth noting:
ProgressType extends JobProgresskeepsJob<…, P>assignable to the internalMinimalJob<…, JobProgress>helpers, which never touchprogress. SoScripts/Backoffsneed no threading and are untouched.ProgressTypesits afterBonWorker. The 4th slot is taken by the v6 backend generic, so adding it 5th avoids breaking the v6 signature. OnJob/MinimalJob/Processorit is purely additive as the 4th.Additional Notes (Optional)
Ordering is open. Cost of the above is that users must name the backend to reach
ProgressType. Happy to move it toWorker's 4th slot instead if you'll take it as a breaking change.Intentionally out of scope:
JobProgress— the value crosses a process boundary where the static type is lost.getJob,Job.fromId/fromJSONkeep the default; threadingProgressTypethrough the getter surface can be a follow-up.progressstill defaults to0, typed asProgressTypewith the initial value cast rather than widened toProgressType | number, which would defeat the generic at every read.Verified:
tsc --noEmitclean on both tsconfigs; eslint, prettier and madge clean;tests/worker.test.ts123/123 and thejob/queue/events/flowsuites 198/198 against Redis 8. Type safety checked with@ts-expect-errorassertions (narrowed type in the processor and theprogress/active/completed/failedhandlers,bigintrejected by the constraint, 0- and 3-generic usage still compiles).Reproduce with
yarn && yarn pretest && npx vitest run --no-file-parallelism tests/worker.test.ts.