Skip to content

perf(worker): do fetch next job in same finished roundtrip [rust] - #4277

Merged
roggervalf merged 9 commits into
masterfrom
perf/enable-fetch-next-in-rust
Jul 14, 2026
Merged

perf(worker): do fetch next job in same finished roundtrip [rust]#4277
roggervalf merged 9 commits into
masterfrom
perf/enable-fetch-next-in-rust

Conversation

@manast

@manast manast commented Jul 12, 2026

Copy link
Copy Markdown
Contributor

Port Impact Checklist

  • Python – does this change need to be ported or documented in the Python library?
  • Elixir – does this change need to be ported or documented in the Elixir library?
  • PHP – does this change need to be ported or documented in the PHP library?
  • Rust – does this change need to be ported or documented in the Rust library?

Why

Enter your explanation here.

How

Enter the implementation details here.

Additional Notes (Optional)

Any extra info here.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR updates the Rust worker execution loop to take advantage of BullMQ’s moveToFinished “fetch next job” capability, reducing per-job Redis roundtrips by chaining directly into the next job when available. It also ensures worker-level rate limiting remains enforced when chaining by including limiter options in the packed moveToFinished opts.

Changes:

  • Add limiter to moveToFinished packed options so fetchNext honors rate limiting.
  • Allow move_to_finished_fast / job completion & failure handlers to return an optionally fetched next Job.
  • Update the worker loop to process chained jobs in a tight loop using the same token, avoiding extra moveToActive calls.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread rust/src/worker.rs
@manast
manast requested a review from Copilot July 13, 2026 10:41

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 1 out of 1 changed files in this pull request and generated 2 comments.

Comment thread rust/src/worker.rs
Comment thread rust/src/worker.rs

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

Comment thread rust/src/worker.rs Outdated

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

Comment thread rust/src/worker.rs Outdated
@manast
manast requested a review from Copilot July 13, 2026 22:19

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated no new comments.

@roggervalf
roggervalf merged commit 677e259 into master Jul 14, 2026
26 checks passed
@roggervalf
roggervalf deleted the perf/enable-fetch-next-in-rust branch July 14, 2026 02:39
@github-actions

Copy link
Copy Markdown
Contributor

🎉 This PR is included in version 1.0.1 🎉

The release is available on:

Your semantic-release bot 📦🚀

@github-actions

Copy link
Copy Markdown
Contributor

🎉 This PR is included in version 5.80.3 🎉

The release is available on:

Your semantic-release bot 📦🚀

@github-actions

Copy link
Copy Markdown
Contributor

🎉 This PR is included in version 2.25.4 🎉

The release is available on:

Your semantic-release bot 📦🚀

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants