fix(heartbeat): eagerly cancel a former assignee's stale queued runs on reassignment - #2
Closed
rendedennis6-byte wants to merge 1 commit into
Closed
fix(heartbeat): eagerly cancel a former assignee's stale queued runs on reassignment#2rendedennis6-byte wants to merge 1 commit into
rendedennis6-byte wants to merge 1 commit into
Conversation
…on issue reassignment Reassigning an issue away from an agent left that agent's queued continuation/recovery run for the issue in place. The run only got swept by the lazy staleness check inside claimQueuedRun, which requires the queue to reach that specific run AND a concurrency slot to be free - otherwise the run, and any execution lock it holds, could be stranded for hours. Adds heartbeatService.cancelStaleQueuedRunsForIssue(), called from the issue-update route right when assigneeAgentId changes, and also sweeps the whole queued-run list for staleness up front in the claim path (before the concurrency-slot gate) so a stale entry buried behind live work no longer gets skipped indefinitely. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Owner
Author
|
Closing as duplicate: this fix (eager cancellation of a former assignee's stale queued runs on issue reassignment) was already implemented and submitted upstream as paperclipai#11085 (paperclipai#11085), opened from this same fork slightly earlier. This PR was accidentally opened against my own fork's master instead of upstream, redoing the same work in a separate branch. Please refer to paperclipai#11085 for review/merge. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Thinking Path
Linked Issues or Issue Description
No public GitHub issue exists for this (tracked internally). Describing per the bug report template:
Bug: Reassigning an issue away from its current assignee agent does not cancel that agent's still-
queuedcontinuation/recovery run for the issue. The run is only reconciled by a lazy staleness check inside the claim path (claimQueuedRun), which only runs once (a) the agent's own queue-processing loop reaches that specific run, and (b) the agent has a free concurrency slot. If the agent is at its concurrency cap, or the stale run is queued behind other live/ready work for that agent, the stale run - and any execution lock it may still hold on the issue - can remain stranded indefinitely (observed: hours).Expected: Reassigning an issue should promptly cancel any queued run the previous assignee was still holding for that issue, freeing any lock it holds so the new assignee (or future claims) aren't blocked.
Actual: The stale queued run lingers until the lazy check happens to reach it, which is not guaranteed to happen in a timely manner.
What Changed
heartbeatService.cancelStaleQueuedRunsForIssue(companyId, issueId, previousAssigneeAgentId), which loads the previous assignee'squeuedruns for the given issue, evaluates each with the existingevaluateQueuedRunStalenesslogic (so interaction-wake / current-review-participant exceptions still apply), and cancels the ones found stale.server/src/routes/issues.ts: wheneverassigneeAgentIdchanges during an update, the previous assignee's queued runs for that issue are eagerly swept (best-effort, logged on failure, does not block the response).heartbeat.ts's queued-run claim path, moved the staleness sweep to run over the entire queued list up front, before the concurrency-slot gate, instead of only reconciling runs the claim loop happens to walk past while slots remain. This closes the gap where a stale run sitting behind enough live work (or arriving after the agent is already at its concurrency cap) would never get re-examined.issue-agent-mutation-ownership-routes.test.ts,issue-comment-reopen-routes.test.ts,issue-execution-policy-routes.test.ts,issue-update-comment-wakeup-routes.test.ts) to include the newcancelStaleQueuedRunsForIssuemock onmockHeartbeatService, since the route code now calls it unconditionally on assignee changes.Verification
pnpm -r typecheck(repo-wide): passes clean.pnpm run build(repo-wide): passes clean.serverpackagevitest run(409 files / 4340 tests): 381 files / 4183 tests pass. The 28 failing files are all pre-existing, unrelated to this change - adapter/CLI execution tests (Claude/Codex/Gemini/Cursor/Pi local adapters) and workspace-runtime/environment-runtime tests that require external binaries not present in this shell (sshd,taskkill, adapter CLIs), which is exactly the class of suite the repo'sscripts/run-vitest-stable.mjswrapper normally serializes/excludes from a plainvitest run. None of the 4 edited test files appear in that failure list.denies company-wide issue list routes for task bridge keysinissue-agent-mutation-ownership-routes.test.ts, a 5000ms timeout) was confirmed pre-existing and unrelated by re-running it against the unmodified base branch (git stash), where it fails identically.evaluateQueuedRunStaleness/cancelQueuedRunForStaleIssueto confirm the eager sweep reuses the same staleness rules (including the interaction-wake / review-participant exceptions) as the existing lazy check, rather than introducing a second, divergent cancellation policy.Risks
.catchthat logs a warning rather than failing the issue-update request, so a transient DB error here cannot block a legitimate reassignment.Model Used
Claude Sonnet 4.6, via Claude Code (agentic coding tool with file read/write, bash execution, and test running tool use).
Checklist
Fixes: #/Closes #/Refs #OR (b) described the issue in-PR following the relevant issue template#NNN/github.com/paperclipai/paperclipURLs)docs/...,fix/...) and contains no internal Paperclip ticket id or instance-derived details