fix(sources): paginate github issues on the gateway raw page length - #176
Merged
Conversation
The OC list_repository_issues action filters pull requests out AFTER paginating, so the filtered issues array's length is not a termination signal: short-page termination silently truncated the scan on the first PR-bearing page, and even empty-page termination fails on 100 consecutive PRs. Upstream now reports the raw page length (pageInfo.fetched, oomol-lab/open-connector#228); this change consumes it. Engine: PageNumber gains raw_page_size_path, mirroring total_pages_path — when declared, the scan continues while the RAW page was full regardless of how short (or empty) the filtered rows are; a missing signal propagates as RowPathNotFound and a non-integer one fails as the new PaginationRawPageSizeInvalid (kind-only), never a silent truncation. The two authoritative signals are mutually exclusive at validate time. Loader exposes raw_page_size_path on page_number pagination; the issues table declares $.pageInfo.fetched. Contract re-captured from a live gateway carrying the upstream fix (outputSchema now declares pageInfo) and the fingerprint re-pinned, so older gateways fail issues registration at the fingerprint gate instead of truncating. Live-verified: all 11 tables register with the new pin and the issues scan reaches the credential wall. Tests: engine units (continue on full raw page with short/empty rows, terminate on short raw page, missing/invalid signal failures, total/raw mutual exclusion) and a pack e2e driving a 3-page scan whose middle page is ALL pull requests — the case that defeats every filtered-count heuristic. Stubs and the demo stub gateway now emit pageInfo. Docs and spec updated with the minimum-gateway note. 246 open_connector / 803 lib tests green. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
BtXin
approved these changes
Jul 30, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Closes the deferred P1 on
github.issues: the OC action filters pull requests out after paginating, so the filtered page length is not a termination signal — short-page termination silently truncated on the first PR-bearing page, and even empty-page termination fails on 100 consecutive PRs. Upstream now reports the raw page length (open-connector#228, merged); this PR consumes it.PageNumbergainsraw_page_size_path(mirroringtotal_pages_path) — the scan continues while the RAW page was full, regardless of how short or empty the filtered rows are. Missing signal →RowPathNotFound; non-integer → newPaginationRawPageSizeInvalid(kind-only). The two authoritative signals are mutually exclusive at validate time.github.yamlissues declaresraw_page_size_path: "$.pageInfo.fetched"; contract re-captured from a live gateway carrying the fix and the fingerprint re-pinned — older gateways fail issues registration at the fingerprint gate instead of truncating (documented, incl. the minimum-gateway note).pageInfo.246 open_connector / 803 lib tests green; clippy baseline unchanged.
🤖 Generated with Claude Code