feat(job-options): warn on suspiciously-large removeOn*.age values (#3540) - #4059
Open
maruthang wants to merge 2 commits into
Open
feat(job-options): warn on suspiciously-large removeOn*.age values (#3540)#4059maruthang wants to merge 2 commits into
maruthang wants to merge 2 commits into
Conversation
…askforcesh#3540) Issue taskforcesh#3540 reports jobs being auto-removed without explanation. The reporter (and others in similar reports) configured retention with values such as `7 * 24 * 60 * 60 * 1000`, mistakenly assuming the `age` field is expressed in milliseconds. BullMQ interprets it as seconds, leading to nonsensical retention windows of ~19 years and masking the original intent. Add a non-throwing runtime warning emitted from Queue and Worker construction (and from `Queue#add`'s merged options) when `removeOnComplete.age` or `removeOnFail.age` exceeds 10 years (the documented `MAX_REASONABLE_KEEP_JOBS_AGE_SECONDS`). The warning is deduplicated per (context, value) so noisy applications only see it once. Also tighten the JSDoc on `KeepJobs.age` and the surrounding option fields to make the seconds-vs-milliseconds distinction explicit, with worked examples for 7-day and 30-day retention.
…moveOn-age-validation # Conflicts: # src/classes/queue.ts # src/classes/worker.ts
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
Closes #3540. The reporter observed jobs being unexpectedly auto-removed with no logs, with the most likely root cause being a unit-confusion bug:
removeOnComplete.ageandremoveOnFail.ageare documented in seconds, but it's easy to pass a value computed in milliseconds (e.g.7 * 24 * 60 * 60 * 1000instead of7 * 24 * 60 * 60). Treated as seconds, those values become decades or centuries — the user thinks "retention 7 days", actually configures "retention ~19 years", and is surprised when behavior diverges from expectations. The current code silently accepts them.How
validateKeepJobsAgeinsrc/utils/index.tsand exportedMAX_REASONABLE_KEEP_JOBS_AGE_SECONDS = 315_360_000(10 years). Whenageexceeds the threshold, a non-throwingconsole.warnis emitted suggesting the user may have confused milliseconds with seconds and showing the equivalent value in days/years.Queueconstructor (defaultJobOptions)Workerconstructor (removeOnComplete/removeOnFail)Queue#addJobmerged optsKeepJobs.age,DefaultJobOptions.removeOn*, andWorkerOptions.removeOn*to make the seconds unit explicit and include a worked example:7 * 24 * 60 * 60.Additional Notes
tests/validate_keep_jobs_age.test.ts(10 unit tests, all passing);tests/clean.test.ts(42 Redis-backed) all green — no regression in the option-parsing path.yarn buildandyarn lintclean.