feat: wake up Promotions waiting on PRs via pull_request webhooks#5852
feat: wake up Promotions waiting on PRs via pull_request webhooks#5852DavidS-ovm wants to merge 56 commits intoakuity:mainfrom
Conversation
✅ Deploy Preview for docs-kargo-io ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
6c5e9c3 to
921c263
Compare
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## main #5852 +/- ##
==========================================
+ Coverage 57.03% 57.33% +0.29%
==========================================
Files 463 470 +7
Lines 39112 39961 +849
==========================================
+ Hits 22309 22911 +602
- Misses 15474 15681 +207
- Partials 1329 1369 +40 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
pkg/indexer/indexer.go
Outdated
| func RunningPromotionsByPullRequest( | ||
| ctx context.Context, | ||
| cl client.Client, | ||
| ) client.IndexerFunc { |
There was a problem hiding this comment.
We might be doing a lot more than is necessary to build this index. I think all the expression evaluation stuff probably isn't necessary. The PR URL is in a git-open-pr step's output. That value will always be static; not an expression. Isn't that sufficient information for inferring a relationship between a PR and a Promotion?
There was a problem hiding this comment.
This was an interesting one. I was under the impression that running steps didn't have outputs yet (but then I only looked at our 1.8 install, and maybe just the wrong steps, or it doesn't show up in the app). Since then I've updated to 1.9 and looked at our git-wait-for-pr step, and indeed, the required values are right there! So I removed the whole index and replaced it with a small check of those outputs.
This might have a small window where github is fast enough to deliver an event but the step hasn't yet updated - maybe if people are doing weird stuff in the stage between opening the PR and waiting for the merge, but in that case either the task already sees the merged PR and proceeds, or - worst case - the polling loop will catch it later.
There was a problem hiding this comment.
It would be good if we didn't do this only for GitHub, but for all git hosting providers that support a similar event.
There was a problem hiding this comment.
I agree that that would be good.
I don't have infrastructure to test, nor budget to support anything other than what my team uses.
|
I have this an initial look, but wondering if @fuskovic would be so kind as to take over as the sherpa here. |
Add an indexer that maps running Promotions to the pull requests they
are waiting on via git-wait-for-pr steps. Index keys use the form
"{normalizedRepoURL}:{prNumber}", enabling efficient lookup when a
webhook arrives for a specific PR.
The indexer follows the existing RunningPromotionsByArgoCDApplications
pattern: filtering to running Promotions, iterating steps up to
CurrentStep, evaluating template expressions for repoURL and prNumber,
and normalizing Git URLs. It is registered in both the promotion
controller and the external webhook server.
Signed-off-by: David Schmitt <david.schmitt@overmind.tech>
Extend the GitHub webhook receiver to accept pull_request events. On "closed" action (merged or not), look up running Promotions waiting on the PR via the RunningPromotionsByPullRequestField index and call RefreshObject on each to trigger immediate re-reconciliation. Non-closed actions return 200 OK with no side effects. The new refreshPromotions helper in refresh.go mirrors refreshWarehouses: it queries both clone URL and SSH URL forms, de-dupes by ObjectKey, and is provider-agnostic so future GitLab/Bitbucket handlers can reuse it. Signed-off-by: David Schmitt <david.schmitt@overmind.tech>
…tions Update the GitHub webhook receiver page to describe the new pull_request event handling and add pull_request to the event selection instructions across all three setup methods (single repo, org, GitHub App). Add a webhook acceleration tip to the git-wait-for-pr step reference. Broaden the webhook receivers index page, Working with Warehouses guide, and Cluster Level Configuration guide to mention Promotion refresh alongside Warehouse discovery. Signed-off-by: David Schmitt <david.schmitt@overmind.tech>
The function takes a PR number as a use-case-specific argument, unlike refreshWarehouses whose parameters are intrinsic to any Warehouse. Using a more specific name leaves room for future refreshPromotionsBy*() functions triggered by other events (e.g. deployment health checks, external approvals). Signed-off-by: David Schmitt <david.schmitt@overmind.tech>
Replace the expression-evaluation-based RunningPromotionsByPullRequest indexer with a simpler version that reads PR data directly from git-wait-for-pr step outputs in promo.Status.State. The previous implementation (~170 lines) fetched Stage objects, built promotion contexts, created step evaluators, and template-evaluated repoURL/prNumber config fields. This was over-engineered and had a latent bug: freightMetadata() in config fields (not vars) could not be evaluated because the indexer did not pass the required expr options. The new implementation (~40 lines) reads the already-resolved PR data (pr.id, pr.url) from each git-wait-for-pr step's output in State. The step writes this output on every reconciliation cycle, even when the PR is still open, making expression evaluation unnecessary. This also removes the ctx/client.Client parameters from the function signature, making it a plain client.IndexerFunc like other indexers. This also removes the Stage informer registration that is not needed anymore. Signed-off-by: David Schmitt <david.schmitt@overmind.tech>
… lookup Remove the RunningPromotionsByPullRequestField index registration from the promotion controller — it is only queried by the external webhooks server and adds unnecessary memory and CPU overhead to every Promotion status update. Remove the SSH URL from the pull_request webhook handler's index lookup. The index keys are always built from the HTTPS repoURL literal in git-wait-for-pr step config, so the SSH form never matches. Signed-off-by: David Schmitt <david.schmitt@overmind.tech>
Signed-off-by: fuskovic <fhuskovic92@gmail.com>
Signed-off-by: fuskovic <fhuskovic92@gmail.com>
Signed-off-by: fuskovic <fhuskovic92@gmail.com>
Signed-off-by: fuskovic <fhuskovic92@gmail.com>
Signed-off-by: fuskovic <fhuskovic92@gmail.com>
Signed-off-by: fuskovic <fhuskovic92@gmail.com>
… accordingly Signed-off-by: fuskovic <fhuskovic92@gmail.com>
Signed-off-by: fuskovic <fhuskovic92@gmail.com>
Signed-off-by: fuskovic <fhuskovic92@gmail.com>
Signed-off-by: fuskovic <fhuskovic92@gmail.com>
Signed-off-by: fuskovic <fhuskovic92@gmail.com>
Signed-off-by: fuskovic <fhuskovic92@gmail.com>
Signed-off-by: fuskovic <fhuskovic92@gmail.com>
Signed-off-by: fuskovic <fhuskovic92@gmail.com>
Signed-off-by: fuskovic <fhuskovic92@gmail.com>
Signed-off-by: fuskovic <fhuskovic92@gmail.com>
Signed-off-by: fuskovic <fhuskovic92@gmail.com>
Signed-off-by: fuskovic <fhuskovic92@gmail.com>
Signed-off-by: fuskovic <fhuskovic92@gmail.com>
Signed-off-by: fuskovic <fhuskovic92@gmail.com>
Signed-off-by: fuskovic <fhuskovic92@gmail.com>
…rver Signed-off-by: fuskovic <fhuskovic92@gmail.com>
Signed-off-by: fuskovic <fhuskovic92@gmail.com>
Signed-off-by: fuskovic <fhuskovic92@gmail.com>
Summary
When a GitHub
pull_requestevent with aclosedaction arrives, Kargo can nowimmediately re-reconcile any running Promotion whose current
git-wait-for-prstep is waiting on that PR — reducing detection latency from ~5 minutes (polling)
to near-instant. Polling remains as a reliability fallback.
Closes #5834. Related: #3303, #647, #4969, #5450
Design
This follows the approach discussed in
#5450 (comment) —
extending the existing webhook infrastructure rather than introducing a new
mechanism.
Indexer — A new
RunningPromotionsByPullRequestFieldindexer maps{normalizedRepoURL}:{prNumber}to running Promotions. It follows theRunningPromotionsByArgoCDApplicationspattern: filtering to runningPromotions, iterating steps up to
CurrentStep, evaluating templateexpressions for
repoURLandprNumber, and normalizing Git URLs. Registeredin the external webhook server.
Webhook handler — The GitHub receiver now accepts
pull_requestevents.On
closedaction (merged or not), it queries the index with both the cloneURL and SSH URL from the event payload, de-dupes matching Promotions by
ObjectKey, and callsapi.RefreshObjecton each. Non-closed actions return200 OK with no side effects. The new
refreshPromotionshelper inrefresh.gomirrors
refreshWarehousesand is provider-agnostic so future GitLab/Bitbuckethandlers can reuse it.
Why only
closed? A merged PR firesclosedwithmerged: true; thegit-wait-for-prstep detects the merge on next reconcile and succeeds. A PRclosed without merge fires
closedwithmerged: false; the step detectsclosure and fails with a terminal error. All other actions (
opened,synchronize, etc.) are irrelevant to the step's outcome.User-facing changes
pull_requestto the event types on their GitHub webhookconfiguration (one-time change). Users who do not make this change experience
no behavior change.
Documentation
pull_requestto supported events, addedPromotion "refreshing" explanation, updated event selection instructions for
all three setup methods (single repo, org, GitHub App)
git-wait-for-prstep reference: added a tip about webhook accelerationConfiguration guide: broadened webhook framing to mention Promotion refresh
Test plan
literal values, non-
git-wait-for-prsteps ignored, non-running Promotionsexcluded, multiple PRs across steps, URL normalization (clone URL, SSH URL),
template expressions with outputs and vars, steps beyond
CurrentStepPromotion, closed+not-merged refreshes Promotion, non-closed action returns
200, no matching Promotions returns 200 with 0 refreshed, SSH URL form matches
Signed-off-by: David Schmitt david@overmind.tech