Skip to content

feat: add generic ProgressType parameter to Job and Worker - #4529

Open
skrukwa wants to merge 2 commits into
taskforcesh:masterfrom
skrukwa:feat/generic-progress-type
Open

feat: add generic ProgressType parameter to Job and Worker#4529
skrukwa wants to merge 2 commits into
taskforcesh:masterfrom
skrukwa:feat/generic-progress-type

Conversation

@skrukwa

@skrukwa skrukwa commented Aug 7, 2026

Copy link
Copy Markdown

The progress type was hardcoded to JobProgress
(string | boolean | number | object), so there was no way to get type-safe progress when calling updateProgress() or when listening to the progress event.

Add an optional ProgressType generic parameter, constrained to JobProgress and defaulting to JobProgress, to MinimalJob, Job, Processor, Worker and WorkerListener. Existing code that passes fewer generics keeps compiling unchanged.

Because ProgressType is constrained to JobProgress, Job<..., P> stays assignable to the internal MinimalJob helpers, which never read or write progress, so the generic does not need to be threaded through them. This also keeps the persistence contract intact, since progress is ultimately serialized before being stored in the backend.

On Worker the parameter is added after the backend generic B so that the existing public signature is not broken.

Sandboxed processors keep JobProgress by design, as progress crosses a process boundary where the static type is lost. Queue side getters such as getJob and Job.fromId also keep the default.

Closes #3721


Port Impact Checklist

  • Python – does this change need to be ported or documented in the Python library?
  • Elixir – does this change need to be ported or documented in the Elixir library?
  • PHP – does this change need to be ported or documented in the PHP library?
  • Rust – does this change need to be ported or documented in the Rust library?
  • .NET – does this change need to be ported or documented in the .NET library?

TypeScript types only — no runtime or Lua changes, nothing to port.

Why

JobProgress is hardcoded as string | boolean | number | object, so progress can't be type-safe:

worker.on('progress', (job, progress) => {
  progress.percentage; // TS error: Property 'percentage' does not exist on type 'JobProgress'
});

Closes #3721 (also discussed in #3184).

Continues #3909 by @claygeo, closed when the head repo was deleted. Rebased onto current master and reworked for the v6 backend generic. @claygeo credited as co-author.

How

Add an optional ProgressType generic to MinimalJob, Job, Processor, Worker and WorkerListener, constrained extends JobProgress and defaulting to JobProgress. Existing code with fewer generics compiles unchanged.

type MyProgress = { percentage: number; message: string };

const worker = new Worker<MyData, MyReturn, string, RedisQueueBackend, MyProgress>(
  'myQueue',
  async job => {
    await job.updateProgress({ percentage: 50, message: 'halfway' }); // type-safe
  },
  { connection },
);

worker.on('progress', (job, progress) => {
  console.log(progress.percentage, progress.message); // number, string
});

Two decisions worth noting:

  • The constraint replaces internal plumbing. ProgressType extends JobProgress keeps Job<…, P> assignable to the internal MinimalJob<…, JobProgress> helpers, which never touch progress. So Scripts/Backoffs need no threading and are untouched.
  • ProgressType sits after B on Worker. The 4th slot is taken by the v6 backend generic, so adding it 5th avoids breaking the v6 signature. On Job/MinimalJob/Processor it is purely additive as the 4th.

Additional Notes (Optional)

Ordering is open. Cost of the above is that users must name the backend to reach ProgressType. Happy to move it to Worker's 4th slot instead if you'll take it as a breaking change.

Intentionally out of scope:

  • Sandboxed processors keep JobProgress — the value crosses a process boundary where the static type is lost.
  • getJob, Job.fromId/fromJSON keep the default; threading ProgressType through the getter surface can be a follow-up.
  • progress still defaults to 0, typed as ProgressType with the initial value cast rather than widened to ProgressType | number, which would defeat the generic at every read.

Verified: tsc --noEmit clean on both tsconfigs; eslint, prettier and madge clean; tests/worker.test.ts 123/123 and the job/queue/events/flow suites 198/198 against Redis 8. Type safety checked with @ts-expect-error assertions (narrowed type in the processor and the progress/active/completed/failed handlers, bigint rejected by the constraint, 0- and 3-generic usage still compiles).

Reproduce with yarn && yarn pretest && npx vitest run --no-file-parallelism tests/worker.test.ts.

Copilot AI review requested due to automatic review settings August 7, 2026 18:41

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR extends BullMQ’s TypeScript surface to support a user-defined, type-safe job progress shape by introducing an optional ProgressType generic (constrained to JobProgress and defaulting to JobProgress) across MinimalJob, Job, Processor, Worker, and WorkerListener, and documents/validates the new typing via docs and tests.

Changes:

  • Add ProgressType extends JobProgress = JobProgress to MinimalJob and Job, typing progress and updateProgress() accordingly.
  • Thread ProgressType through Worker and WorkerListener so worker.on('progress', ...) and processor job.updateProgress(...) become type-safe.
  • Add docs + a worker test demonstrating typed progress usage.

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
tests/worker.test.ts Adds a test covering typed progress generics on Worker and the progress event.
src/types/processor.ts Adds a 4th generic parameter to Processor so processor functions get typed job.updateProgress().
src/interfaces/minimal-job.ts Adds ProgressType generic and types progress/updateProgress() on the minimal job contract.
src/classes/worker.ts Threads ProgressType through Worker and WorkerListener event typing and processor typing.
src/classes/job.ts Adds ProgressType generic and types progress field + updateProgress().
docs/gitbook/guide/workers/README.md Documents how to use the new ProgressType generic with Worker.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread tests/worker.test.ts Outdated

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 6 out of 6 changed files in this pull request and generated no new comments.

Suppressed comments (3)

src/classes/job.ts:70

  • Job.progress is typed as ProgressType, but it is initialized to the numeric default 0. If a user supplies a non-numeric ProgressType (e.g. an object shape), TypeScript will allow property access on job.progress that will fail at runtime until the first updateProgress() call. Consider reflecting the persisted default by including 0 in the type instead of casting.
  progress: ProgressType = 0 as ProgressType;

src/classes/worker.ts:405

  • createSandbox is only used for sandboxed processors, which intentionally keep JobProgress because progress crosses a process boundary. Having the sandboxed-processor overload accept Processor<..., ProgressType> is misleading for subclasses and suggests typed progress is preserved in sandbox mode. Consider using the default Processor<..., JobProgress> form here (omit the ProgressType parameter).
      | Processor<DataType, ResultType, NameType, ProgressType>,

src/interfaces/minimal-job.ts:80

  • MinimalJob.progress is declared as ProgressType, but BullMQ initializes progress to 0. For custom progress shapes this makes the public type unsound (progress can be a number before the first update). Align the interface with runtime by allowing the default 0 value in the type.
  progress: ProgressType;

The progress type was hardcoded to JobProgress
(string | boolean | number | object), so there was no way to get
type-safe progress when calling updateProgress() or when listening to
the progress event.

Add an optional ProgressType generic parameter, constrained to
JobProgress and defaulting to JobProgress, to MinimalJob, Job,
Processor, Worker and WorkerListener. Existing code that passes fewer
generics keeps compiling unchanged.

Because ProgressType is constrained to JobProgress, Job<..., P> stays
assignable to the internal MinimalJob helpers, which never read or
write progress, so the generic does not need to be threaded through
them. This also keeps the persistence contract intact, since progress
is ultimately serialized before being stored in the backend.

On Worker the parameter is added after the backend generic B so that
the existing public signature is not broken.

Sandboxed processors keep JobProgress by design, as progress crosses a
process boundary where the static type is lost. Queue side getters such
as getJob and Job.fromId also keep the default.

Closes taskforcesh#3721

Co-authored-by: claygeo <claygeo6@gmail.com>
@skrukwa
skrukwa force-pushed the feat/generic-progress-type branch from b1f4e62 to 30c4eea Compare August 7, 2026 19:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature]: add generic type for job progress

2 participants