fix: reduce executable registry size - #237
Conversation
Summary by CodeRabbit
WalkthroughThe provider registry generator now emits only Sequence Diagram(s)sequenceDiagram
participant Server
participant executorModules
participant CatalogLoader
participant CatalogStore
Server->>executorModules: read service keys
Server->>CatalogLoader: pass executableServices
CatalogLoader->>CatalogLoader: resolve service actions and explicit IDs
CatalogLoader->>CatalogStore: create catalog with executableActionIds
Possibly related issues
🚥 Pre-merge checks | ✅ 3 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (3 passed)
✨ Finishing Touches✨ Simplify code
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
🧹 Nitpick comments (1)
src/catalog-store.test.ts (1)
76-88: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winAdd a negative service-selection assertion.
The fixture currently makes every action executable, so it would not catch a regression that marks all catalog actions as locally executable. Add an unselected action and assert it remains catalog-only.
Suggested test adjustment
- const providers = [providerFixture("example", ["ping", "pong"]), providerFixture("remote", ["ping"])]; + const providers = [providerFixture("example", ["ping", "pong"]), providerFixture("remote", ["ping", "pong"])]; expect(catalog.executableActionIds).toEqual(new Set(["example.ping", "example.pong", "remote.ping"])); expect(catalog.actionsById.get("example.pong")?.execution.locallyExecutable).toBe(true); + expect(catalog.executableActionIds.has("remote.pong")).toBe(false); + expect(catalog.actionsById.get("remote.pong")?.execution.locallyExecutable).toBe(false);🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/catalog-store.test.ts` around lines 76 - 88, Update the test case in “resolves every action from executable services alongside explicit action ids” to include an action from a non-selected service, then assert that action remains in the catalog but is not marked locally executable. Keep the existing executable-service and explicit-action assertions unchanged.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Nitpick comments:
In `@src/catalog-store.test.ts`:
- Around line 76-88: Update the test case in “resolves every action from
executable services alongside explicit action ids” to include an action from a
non-selected service, then assert that action remains in the catalog but is not
marked locally executable. Keep the existing executable-service and
explicit-action assertions unchanged.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro Plus
Run ID: cd0beeec-9d1a-4fb8-b30e-a359ec29040a
📒 Files selected for processing (6)
scripts/generate-provider-registry.tssrc/catalog-store.test.tssrc/catalog-store.tssrc/server/cloudflare.tssrc/server/cloudflare/catalog-assets.tssrc/server/index.ts
💤 Files with no reviewable changes (1)
- scripts/generate-provider-registry.ts
## Summary - replace the monolithic Cloudflare `/catalog/apps.json` build output with a versioned `/catalog/index.json` - write deterministic, sequential catalog chunks capped at 4 MiB by their final UTF-8 byte size - load all indexed chunks through the existing `ASSETS` binding while preserving `CatalogStore` and public API response shapes - retain a loader fallback to `/catalog/apps.json` for deployments using the previous asset format - request catalog assets as JSON so a missing index is not treated as an SPA navigation This is the catalog follow-up requested in #235 after the executable registry reduction merged in #237. ## Why On the current `main` catalog (1,210 providers and 12,894 actions), the generated `apps.json` is 26,030,332 bytes (24.8245 MiB), leaving only 184,068 bytes before Cloudflare's 25 MiB per-asset limit. The provider catalog already consists of individual source files, so combining everything into one deployment asset creates an avoidable single-file limit. ## Cloudflare build comparison Measured on the same current `main` catalog with Wrangler 4.115.0, `--dry-run`, and `--minify`: | Measurement | Before | This PR | | --- | ---: | ---: | | Largest catalog asset | 26,030,332 bytes (24.8245 MiB) | 4,192,730 bytes (3.9985 MiB) | | Catalog asset files | 1 | 8 (1 index + 7 chunks) | | Total static assets | 11 | 18 | | Worker minified upload | 11,672.94 KiB | 11,674.91 KiB | | Worker gzip | 2,805.33 KiB | 2,806.08 KiB | New builds do not emit the complete `apps.json`. The small Worker increase is the index validation, compatibility fallback, and chunk loading logic. ## Compatibility and validation - provider files are sorted by filename before chunking for deterministic output - chunk limits include JSON brackets, commas, trailing newline, and UTF-8 multibyte characters - a provider too large for one chunk fails the build with its filename and byte count - index version, fields, chunk names, duplicate names, chunk shapes, and provider count are validated before creating the catalog - indexed chunks are loaded in parallel and flattened in index order - a missing `index.json` falls back to the legacy `apps.json`; new builds emit only the indexed format - existing executable-service resolution from #237 remains unchanged Checks: - `node scripts/generate-catalog.ts` — 1,210 apps / 12,894 actions - `oxlint . --fix` - `oxfmt .` - `node scripts/typecheck.ts src scripts-all examples` - `vitest run` — 61 test files and 613 tests passed - Wrangler dry-run builds for both the current baseline and this branch Closes #235 --------- Co-authored-by: Kevin Cui <bh@bugs.cc>
Summary
CatalogStoreexecutableActionIds, runtime execution metadata, and public response shapesThis is the first of the two changes discussed in #235. Catalog asset chunking will follow in a separate PR.
Why
The generated Cloudflare registry repeated all 12,743 executable action IDs even though each included provider already has a lazy executor module entry. The existing generator marks every action from an included provider executable, so the executor module service keys carry the same information more compactly.
Wrangler size comparison
Measured from
v1.3.3and this branch with Wrangler 4.115.0, the same Cloudflare bindings/configuration,--dry-run, and--minify:v1.3.3baselineThe emitted
cloudflare.jsdecreased from 12,221,596 to 11,843,955 bytes, a reduction of 377,641 bytes.Validation
oxlint . --fixoxfmt .node scripts/typecheck.ts src scripts-all examplesvitest run— 61 test files and 591 tests passedRefs #235