Skip to content

Replace whenever with sidekiq scheduler#14119

Merged
mkllnk merged 5 commits intoopenfoodfoundation:masterfrom
mkllnk:replace-whenever
Apr 1, 2026
Merged

Replace whenever with sidekiq scheduler#14119
mkllnk merged 5 commits intoopenfoodfoundation:masterfrom
mkllnk:replace-whenever

Conversation

@mkllnk
Copy link
Copy Markdown
Member

@mkllnk mkllnk commented Mar 31, 2026 •
edited
Loading

What? Why?

We started using the sidekiq-scheduler when we switched from DelayedJob to Sidekiq. But we never made the effort of moving all scheduled tasks from the whenever schedule to Sidekiq. This PR makes that change.

Depends on:

What should we test?

  • Testing depends on another PR merged in ofn-install.
  • It has been tested locally and I think that a production test is good enough.
  • Check that backups are created every four hours.

Release notes

Changelog Category (reviewers may add a label for the release notes):

  • User facing changes
  • API changes (V0, V1, DFC or Webhook)
  • Technical changes only
  • Feature toggled

The title of the pull request will be included in the release notes.

Dependencies

Documentation updates

mkllnk added 4 commits March 31, 2026 12:34
@mkllnk
Move database clean from cron to Sidekiq scheduler
80a12db 
After moving the remaining tasks from schedule.rb to sidekiq.yml, we can
remove whenever and won't rely on cron any more. That will simplify the
setup and migration to a new server.
@mkllnk
Schedule all jobs with Sidekiq
b61f6ab
@mkllnk
Remove whenever config
60edcad
@mkllnk
Remove unused gem whenever
c74624c
@mkllnk mkllnk self-assigned this Mar 31, 2026
@mkllnk mkllnk added the technical changes only These pull requests do not contain user facing changes and are grouped in release notes label Mar 31, 2026
@github-project-automation github-project-automation Bot moved this to All the things 💤 in OFN Delivery board Mar 31, 2026
@mkllnk mkllnk mentioned this pull request Mar 31, 2026
Merged
@mkllnk mkllnk marked this pull request as ready for review March 31, 2026 04:40
@mkllnk mkllnk moved this from All the things 💤 to Code review 🔎 in OFN Delivery board Mar 31, 2026
dacook
dacook approved these changes Mar 31, 2026
View reviewed changes
Copy link
Copy Markdown
Member

@dacook dacook left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice work!
I liked whenever, but I can see the advantage to be running any task in the background as a job, which can be queued, retried etc if necessary.

Comment thread app/jobs/rake_job.rb
name, _args = Rake.application.parse_task_string(task_string)
Rake::Task[name].reenable
end
end
Copy link
Copy Markdown
Member

@dacook dacook Mar 31, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice way to wrap up the existing tasks 🥟

Comment thread config/sidekiq.yml Outdated
every: "5m"
RakeJob:
args: ["ofn:data:remove_transient_data"]
cron: "30 4 1 * *"
Copy link
Copy Markdown
Member

@dacook dacook Mar 31, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A shame we lose the nice scheduling DSL :(

Copy link
Copy Markdown
Member Author

@mkllnk mkllnk Apr 1, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A shame we lose the nice scheduling DSL :(

Yes, I tried to keep it. But if you say to run it every month then it starts counting after each deploy and never gets there. It would actually be quite easy to add but I didn't bother about that.

rioug
rioug approved these changes Mar 31, 2026
View reviewed changes
Copy link
Copy Markdown
Collaborator

@rioug rioug left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good one ! Good to see a dependency removed.

@rioug rioug moved this from Code review 🔎 to Dev Test Ready 🔧 in OFN Delivery board Mar 31, 2026
@mkllnk mkllnk added the pr-staged-uk staging.openfoodnetwork.org.uk label Apr 1, 2026
@mkllnk
Copy link
Copy Markdown
Member Author

mkllnk commented Apr 1, 2026

I'm testing this on staging and Sidekiq raises an error there:

app/jobs/rake_job.rb:6:in `RakeJob#perform': uninitialized constant Rake::Task

    Rails.application.load_tasks if Rake::Task.tasks.empty?
                                        ^^^^^^ (NameError)
    from gems/activejob-7.1.6/lib/active_job/execution.rb:68:in `block in ActiveJob::Execution#_perform_job'

This doesn't happen locally. I wonder what the difference is. While I don't know how to re-produce it locally, I'll try to fix it on staging.

@mkllnk
Copy link
Copy Markdown
Member Author

mkllnk commented Apr 1, 2026

Ah, running in staging environment I can reproduce.

@mkllnk mkllnk added pr-staged-uk staging.openfoodnetwork.org.uk and removed pr-staged-uk staging.openfoodnetwork.org.uk labels Apr 1, 2026
@mkllnk
Copy link
Copy Markdown
Member Author

mkllnk commented Apr 1, 2026

Adding a require statement solved the problem.

@mkllnk mkllnk merged commit 450fe4a into openfoodfoundation:master Apr 1, 2026
33 checks passed
@mkllnk mkllnk deleted the replace-whenever branch April 1, 2026 04:37
@github-project-automation github-project-automation Bot moved this from Dev Test Ready 🔧 to Done in OFN Delivery board Apr 1, 2026
@AMEA-LYON AMEA-LYON removed the pr-staged-uk staging.openfoodnetwork.org.uk label Apr 2, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@dacook dacook dacook approved these changes

@rioug rioug rioug approved these changes

Assignees

@mkllnk mkllnk

Labels

technical changes only These pull requests do not contain user facing changes and are grouped in release notes

Projects

OFN Delivery board
Status: Done

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@mkllnk @dacook @rioug @AMEA-LYON