github boundary gate: overflow past MAX_BOUNDARY_IDS re-appends the same rows and re-spends requests every tick

[philcunliffe]

Context

Deferred from PR #1330 (head ffb4e260b8da480814ca7679f65d847e21010fda), review round 2, finding 1.
Severity as reviewed: LOW-MEDIUM. Not a regression: origin/master behaves the same or worse in the identical scenario.

Location

• hypaware-core/plugins-workspace/github/src/cursors.js:26 (MAX_BOUNDARY_IDS = 1000) and cursors.js:159-164 (readBoundaryIds cap)
• hypaware-core/plugins-workspace/github/src/capture.js:738 (openGate's ids getter, same slice(0, MAX_BOUNDARY_IDS))

Observed behavior

The boundary gate carries the event ids sitting exactly on a pass's published watermark so an inclusive since cannot re-append them. The carried set is capped at 1000 ids, symmetrically on write and read. When more than 1000 items share one watermark second (a bulk label or milestone sweep, or a history rewrite that restamps thousands of commit dates), the overflow sits outside the guard. slice(0, 1000) keeps insertion order and the published ids are re-seeded first on the next tick, so the same overflow is never absorbed. Reviewer probe at the PR head, with the request budget raised to isolate the effect:

N=1500:  tick1 1500 commit rows, 1500 listCommitFiles   boundary=1000
         tick2  500 commit rows,  500 listCommitFiles   boundary=1000
         tick3  500 ...           tick4  500 ...        no convergence
N=900:   tick1  900 rows          tick2/3/4  0 rows     boundary=900

Under the production 400-request budget, 500 listCommitFiles calls per tick means the repository never finishes its commits phase and starves the other repositories in rotation, in addition to duplicate github_events and commit_file rows.

Why deferring is safe

On origin/master the identical scenario re-appends all 1500 items every tick and starves the budget the same way, with no cap at all. The PR strictly improves the case and fully fixes #1284 for every boundary set that fits the cap. The trigger needs more than 1000 items stamped on one wall-clock second, which normal repository activity does not produce.

The reason it was left unfixed in the PR: the only exits are accepting a bounded loss (advance the watermark past a second that will not fit, losing items added to that second after the read) or accepting an unbounded sidecar. Both fork against the anti-loss trade the gate's doc settles (the trade pullChangedSince also refuses), so the choice needs an LLP design decision, not a review-rung repair. The code comment at cursors.js:145-155 records the loop honestly.

Acceptance condition

An LLP records the chosen exit (bounded loss, unbounded sidecar, or an explicit alternative such as absorbing the overflow across ticks), and with the chosen design in place, the reviewer's probe scenario (1500 commits sharing one committer second, budget not confounding) converges: some tick K appends the remaining rows once and every tick after K appends zero rows and spends zero listCommitFiles requests, with no item published in that second after the first read being silently lost unless the LLP explicitly accepts that loss.

[philcunliffe]

Stuck: escalated to a design request, waiting on the decision

What neutral was doing. The maintenance reconciler admitted this issue for a
fix attempt and read the work item. Its acceptance condition opens with "An LLP
records the chosen exit", so this is an architectural change, not a bug fix, and
the reconciler's rule for that case is to file a request LLP rather than force a
repair.

What was done. Request LLP 0381, "An overflowing watermark second
re-appends its overflow every tick", minted on branch
llp/boundary-overflow-request and proposed in #1368. It states the problem
and the measured probe evidence from PR #1330's round 2, and lays out the three
candidate exits with their costs, choosing none:

• A. Bounded, declared loss - advance the watermark past a second that will
  not fit. Forks against LLP 0361 #page-work's equal-timestamp anti-loss rule
  and the tests pinning it.
• B. Carry the whole second - lift the cap to the second's population.
  Forks against LLP 0360 #resource-bounds, with the precedent that
  pulls_high_numbers and pulls_emitted are already population-bounded.
• C. Absorb across ticks - rotation alone provably does not converge;
  rotation plus a completion marker reduces to a deferred A; overflow-scoped
  downstream dedup is a bounded variant of the option LLP 0373 already rejected
  and still spends the budget every tick.

The shape was checked against LLP 0373 rather than assumed: 0373 sets this
question aside by name in its #constraints section, and because every arm of
0373 carries a boundary set of the same cardinality, all of them overflow
identically. Overflow is orthogonal to the identity-key question, so it is a new
request rather than an extension of that Draft. The one real coupling, that a
fingerprint entry is larger and so prices exit B's sidecar growth, is recorded
in 0381 #relation, and the two documents cross-reference each other.

Why this issue cannot proceed. No fix can be attempted without choosing
among A, B and C on neutral's own authority, across the write and read caps and
the openGate getter together. That is the decision LLP 0381 exists to ask for.
Marking this stuck so the reconciler stops re-admitting it and re-deriving the
same escalation every tick.

What it needs from you. Decide the exit in #1368 (or direct the Designer to
a fourth option). Note the interaction: LLP 0373 is also still Draft and
undecided, and its identity-key choice changes the per-entry size that exit B is
priced against, so deciding 0373 first makes 0381's cost comparison concrete.

How to unstick. Reply with a comment on this issue; neutral monitors this
thread and will re-engage with your guidance on its next tick. This issue stays
open until the chosen exit is implemented and the convergence probe in the
acceptance condition passes.