Skip to content

feat: watermark state-tracking with deterministic prioritized task queuing#166

Open
DerekRoberts wants to merge 8 commits intomainfrom
feat/watermark-pattern
Open

feat: watermark state-tracking with deterministic prioritized task queuing#166
DerekRoberts wants to merge 8 commits intomainfrom
feat/watermark-pattern

Conversation

@DerekRoberts
Copy link
Copy Markdown
Member

@DerekRoberts DerekRoberts commented Apr 12, 2026

Summary

This PR consolidates the two major engine upgrades for the Marketplace release:

  1. Watermark Pattern: Implements persistent state-tracking to avoid redundant processing of old issues/PRs.
  2. Deterministic TaskQueue: Replaces procedural backoff with a priority-aware, budget-managed execution engine.

Key Changes

  • Rate Limit Hardening: RatePriority constants decoupled to resolve circular dependencies.
  • API Proxying: All Octokit and GraphQL calls are now automatically prioritized.
  • Sync Efficiency: Watermarks ensure we only sync items updated since the last successful run.
  • Test Stability: 303 tests green with anchored async promise handling.

Results

  • Resolved 'RatePriority undefined' startup crashes.
  • Resolved 'Promise resolution is still pending' test leaks.
  • Engine is now production-ready for v1 marketplace release.

Copilot AI review requested due to automatic review settings April 12, 2026 20:24
@DerekRoberts DerekRoberts moved this from New to Active in DevOps (NR) Apr 12, 2026
@DerekRoberts DerekRoberts self-assigned this Apr 12, 2026
@DerekRoberts DerekRoberts force-pushed the feat/watermark-pattern branch from 860bc81 to 86924fc Compare April 12, 2026 20:28
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Implements a “watermark” (last successful sync timestamp) using GitHub Actions Cache to support gapless synchronization, and threads an explicit since timestamp through the item discovery/search flow.

Changes:

  • Add watermark utility (getWatermark/saveWatermark) backed by @actions/cache and integrate it into the main sync flow.
  • Update GitHub search query construction to prefer an explicit since timestamp (falling back to a sliding window).
  • Expand unit tests for watermark behavior, query construction, and processor utilities; plus various ESM/export/package/docs adjustments.

Reviewed changes

Copilot reviewed 8 out of 9 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
src/utils/watermark.js New cache-backed watermark read/write utility.
src/index.js Loads watermark at start and saves watermark on successful completion; passes since through to item discovery.
src/github/api.js Adds since-driven updated:>=... search clauses (fallback to hour-based window).
src/rules/add-items.js Plumbs a since parameter into getRecentItems options.
test/utils/watermark.test.mjs Unit tests for watermark cache integration and sanitization behavior.
test/github/since-parameter.test.mjs Verifies getRecentItems query construction when since is provided vs fallback window.
test/github/get-recent-items-rate-limit.test.mjs Updates assertions for updated:>= query syntax.
src/utils/log.js Adds warn() alias forwarding to warning().
src/utils/batch.js Converts to ESM exports and adjusts warning logging call-site.
test/utils/batch.test.mjs Adds coverage for processBatch behavior and the new warn() alias.
src/rules/processors/validation.js Refines type-check logic and adds an early return when only type is specified.
test/processors/ludicrous-coverage.test.js Adds extensive matrix coverage for validation/skip whitelist logic and null handling.
src/rules/processors/unified-rule-processor.js Exports deduplicateActions for testing/consumption.
test/processors/deduplication.test.js Adds unit tests for action deduplication behavior.
src/rules/linked-issues.js Converts to ESM imports/exports.
src/utils/status-tracker.js Converts StatusTracker to an ESM export.
package.json Adds @actions/cache/@actions/core, adds build script + @vercel/ncc, changes Node engine.
package-lock.json Locks new dependencies/devDependencies and updated engine metadata.
action.yml Introduces/updates action metadata (name/inputs/env/runtime).
.github/workflows/pr-sync.yml Broadens push branch patterns for workflow trigger.
README.md Substantial documentation rewrite and updated messaging/badges.
memory/constitution.md Updates engineering/testing standards wording (ESM + testing philosophy).
specs/README.md Updates specs index/numbering list and retroactive spec section.
specs/multi-org-support.md Adds a spec doc describing multi-org support behavior/config.
specs/004-rule-engine-architecture/spec.md Adds rule engine architecture spec doc.
specs/005-column-assignment-rules/spec.md Adds column assignment rules spec doc.
specs/006-sprint-assignment-rules/spec.md Adds sprint assignment rules spec doc.
Comments suppressed due to low confidence (1)

package.json:9

  • engines.node was changed to ">=22.0.0", but the repository documentation (README) and workflows appear to target Node 24, and action.yml declares a Node 20 runtime. With engine-strict=true, this will break installs under Node 20/21 and creates inconsistent support guarantees. Pick a single minimum Node version and apply it consistently across package.json, workflows, and action.yml.
  "engines": {
    "node": ">=24.0.0"
  },

@DerekRoberts DerekRoberts changed the title feat: implement watermark pattern for gapless synchronization (#155) feat: watermark state-tracking with deterministic prioritized task queuing Apr 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Active

Development

Successfully merging this pull request may close these issues.

2 participants