saas: DocumentClassifier + PAYG data model - #6460
Merged
Merged
Conversation
Computes the doc-unit cost of an uploaded file (or multi-file input) under a PricingPolicy. PDF page counts via PDFBox; non-PDFs are bytes-only. Result is max(ceil(pages/perPage), ceil(bytes/perBytes)) clamped to [1, fileUnitCap]. Pure utility — no Spring beans wired into it yet, no schema, no Supabase dependency. Foundational for later PRs that compute charges per upload. PricingPolicy lives as a minimal record now (just the fields the classifier reads). A future entity can replace it with the same field surface plus persistence + lifecycle metadata. Tests cover: single-page PDF, multi-page PDF (page axis dominates), large non-PDF (byte axis dominates), file-unit-cap clamp, empty file, malformed PDF (graceful bytes-only fallback), encrypted PDF, missing content type, extension-based PDF detection, multi-file aggregation, multi-file cap. Fixtures generated programmatically via PDFBox at test time — no committed binary blobs.
dosubot
Bot
added
size:L
Frooodle
requested changes
May 28, 2026
Frooodle
left a comment
Frooodle
left a comment
Member
There was a problem hiding this comment.
A AI review also said "Blocking (1 thing): Fix the dead group-cap in multi-file classify. The Math.min(groupCap, unitsSum) can never bind because each file is already clamped to fileUnitCap before summing - so either delete it (if per-file capping is intended) or rewrite to sum raw per-file units so the group cap actually does something. Either way, replace multiFile_isCappedByFileUnitCapTimesFileCount with a test that genuinely exercises the cap (today it passes even with the cap removed)."
- Drop @Profile("saas") — the classifier is stateless utility code, so any deployment can use it without re-configuring profiles. - Switch the page-count read from PDFBox Loader.loadPDF to jpdfium (PdfDocument.open(Path).pageCount()), which is the codebase standard for cheap PDF metadata reads. - Materialise uploads through TempFileManager so jpdfium gets a Path; the TempFile is AutoCloseable and auto-deletes on close. - Fix the dead group cap in classify(List). The previous implementation clamped per file before summing, so Math.min(groupCap, unitsSum) was a no-op. Now: compute raw per-file units, sum, then clamp the sum at fileUnitCap × file_count. The single-file classify(MultipartFile) path keeps its own clamp. - Replace multiFile_isCappedByFileUnitCapTimesFileCount (passed by accident via per-file clamping) with multiFile_groupCapBindsOnSumOfRawUnits — uses asymmetric file sizes so the only way to hit the expected result is via the group cap on the raw sum. Per-file clamping would produce a detectably-different number. - Add singleFileFileUnitCap_clampsExtremelyLargeInputs to keep coverage of the single-file clamp behaviour explicit. - Test policies now use KiB-scale byte units so allocations stay small.
Member
Author
|
Addressed the AI reviewer's blocking finding too: the group cap in Replaced the previous Also added All three inline comments addressed in 04d44d1. PTAL. |
Bundles the full data foundation for the new billing model. Nothing wires
the entities into application behaviour yet; services and controllers land
in follow-up PRs.
New packages under stirling.software.saas.payg:
- model: 13 enums (JobSource, ProcessType, JobStatus, JobStepStatus,
ArtifactKind, LedgerEntryType, LedgerBucket, ReferenceType,
EntitlementState, FeatureSet, FeatureGate, WalletEngine, CapPeriod,
AutoGroupStrategy).
- policy: PricingPolicy promoted from a record to a JPA entity.
Step limits are keyed by JobSource (WEB/API/PIPELINE/DESKTOP) — same
motivation as the "self-hosted gets a different knob" point: a single
knob per caller surface lets self-hosted carry a different ceiling
without bolting a new column on.
- job: ProcessingJob, ProcessingJobStep, JobArtifactHash (composite key).
- wallet: WalletLedgerEntry, WalletPolicy.
- entitlement: WalletEntitlementSnapshot (composite key; user_id = 0
is the team-wide sentinel since Postgres treats NULL as not-equal-to-NULL
in unique constraints).
- shadow: PaygShadowCharge.
- repository: one repo per entity, plus a lineage-lookup query on
JobArtifactHashRepository and period-spend rollups on WalletLedgerRepository.
V11__saas_payg_model.sql creates all eight new tables and adds three
columns to existing tables (teams.pricing_policy_id,
teams.stripe_customer_id, team_memberships.cap_units). Purely additive.
DocumentClassifier updated to use bean-style getters on the new
PricingPolicy entity. A convenience 4-arg constructor preserves the
record-style call shape the classifier and its tests already use.
PaygEntitiesSmokeTest exercises each entity via the no-arg ctor that JPA
requires + a few getter/setter pairs + composite-key equality, catching
Lombok/annotation regressions without needing a database.
dosubot
Bot
added
size:XXL
![aikido-pr-checks[bot]](https://avatars.githubusercontent.com/in/898896?s=60&v=4){kind=small}
ConnorYoh
changed the title
Member
Author
|
Expanded the scope per discussion — now includes the full PAYG data model (8 new tables + column adds across teams + team_memberships) on top of the DocumentClassifier work. Companion Supabase migration: https://github.com/Stirling-Tools/Stirling-PDF-SaaS/pull/296 (targeting the v3 dev branch). Merge order: SaaS PR first (so v3 Supabase has the schema), then this one. Both are purely additive so either order is safe in dev, but landing the schema first lets the application boot cleanly against a fresh v3 database. Open design decision: step limits are keyed by |
- Fix FK column name: teams(id) → teams(team_id) across all eight new tables. The Supabase teams PK is team_id; the previous version failed on the v3 preview branch with SQLSTATE 42703. - Move teams.pricing_policy_id and teams.stripe_customer_id to a payg_team_extensions sidecar table (mirrors saas_team_extensions). PAYG fields no longer touch the shared teams table, so OSS Hibernate ddl-auto against the proprietary Team entity stays unaffected. - Add PaygTeamExtensions JPA entity + repository over the new sidecar. - Add capUnits field to TeamMembership entity (column already in V11). - Rename JobSource.DESKTOP → DESKTOP_APP and add a class-level javadoc clarifying the enum is caller surface only; deployment context (SaaS vs self-hosted) is encoded at the team / pricing_policy level. - Fix WalletEntitlementSnapshot class doc — the team-wide row uses user_id = 0 (not NULL); the previous wording contradicted the impl. Tests: :saas:test all green.
git add -A in the previous commit swept up untracked build output from frontend/src-tauri/provisioner/target/ that has been sitting in this working tree across the session. These are Cargo build artifacts that should never be in the repo. Removing them with git rm --cached so the files stay on disk locally but leave version control.
stirlingbot
Bot
added
Front End
Both PR review feedback from Connor: 1. step_limits and stripe_price_ids were JSONB columns on pricing_policy. That's structured data with a small known shape — better as typed child tables. Replaced: - pricing_policy_step_limit (policy_id, job_source, step_limit) - pricing_policy_stripe_price (policy_id, currency, stripe_price_id) Java-side stays Map<JobSource, Integer> and Map<String, String> via JPA @ElementCollection + @CollectionTable, so callers don't change. No JSON parsing, queryable directly, FK-cascaded on policy delete. 2. wallet_policy.cap_source_currency duplicated stripe.customers.currency. Dropped the column. cap_source_money stays (preserves the customer's money intent across Stripe price changes); currency comes from the team's Stripe customer at recompute time. Counterpart in Stirling-PDF-SaaS#296.
Same Stripe-as-source-of-truth principle as the cap_source_currency drop: currency lives on stripe.prices.currency, not on our row. - Migration: pricing_policy_stripe_price (policy_id, stripe_price_id), PK (policy_id, stripe_price_id). No currency column. - Entity: PricingPolicy.stripePriceIds becomes Set<String> (was Map<String, String>). Currency-aware lookup happens via stripe.prices.currency through Sync Engine. - Smoke test updated to match the Set shape. Trade-off: subscription-creation logic that picks the right Price for a customer's currency will need stripe.prices available (PR-T4 Sync Engine). The hot path (meter event reporting) doesn't read this table at all, so the dependency only matters at subscription-creation time. Counterpart in Stirling-PDF-SaaS#296.
3 tasks
jbrunton96
previously approved these changes
May 28, 2026
ConnorYoh
enabled auto-merge
May 28, 2026 16:38
…iene Six concerns from the 2026-05-29 review, all in one commit. Defers concern #7 (ddl-auto=update with Flyway) to a separate cleanup PR — it's a pre-existing setting whose flip carries OSS-entity-validation risk that needs its own focused change. #1 (HIGH) content_hash widened to VARCHAR(128) at the source - V11: CHAR(64) → VARCHAR(128) (with comment explaining the type:value storage key encoding) - Entity JobArtifactHashId.contentHash: length=64 → 128 + javadoc - Folds the change from #6464's stacked "schema fix" commit down here so #6460 is internally consistent and the Supabase #296 migration's "counterpart to the V11 widen" stops being aspirational. After this lands, the corresponding edit on #6464 becomes a no-op and gets squashed during rebase. #2 (HIGH) Restore @Profile("saas") on DefaultDocumentClassifier - The earlier removal was based on a hypothetical "future paid deployment without the saas profile". Concrete consistency with the 38 other @Profile("saas") beans in the module wins over speculative flexibility. - Class javadoc now documents the choice and the path to broadening (@Profile({"saas", "selfhosted-payg"})) when PR-X1 lands. #3 (MEDIUM, but materially worse than the review noted) Wire payg.* into the JPA scan paths - SaasJpaConfig was scanning saas.repository / .billing.repository / .ai.repository for repos and saas.model / .billing.model / .ai.model for entities — NOT saas.payg.*. The entire PAYG layer was inert: repos weren't beans, entities weren't managed types, every @Autowired PaygSomethingRepository would have failed at startup. Mockito-based unit tests never noticed. - Knock-on: my own stacked PR #6469 (PricingPolicyService) wouldn't have started either. Fixing here fixes both. - Added saas.payg.repository to @EnableJpaRepositories.basePackages and saas.payg (covers payg.policy / payg.job / payg.wallet / payg.entitlement / payg.shadow recursively) to @EntityScan. - New SaasJpaConfigScanTest reads the annotations via reflection and asserts every expected package is present — catches the next time someone adds a payg.X sub-package without wiring the scan. Reflection rather than @DataJpaTest because the production schema uses partial unique indexes that H2 doesn't fully support; standing up Testcontainers for one guard test is disproportionate. #4 (MEDIUM) Document the minChargeUnits flow - Per design § 3.4 the charge formula is unitsForProcess = max(policy.min_charge_units, docUnits), applied at process-open time in JobChargeService — NOT at classify time. The classifier's hardcoded floor of 1 is a separate "non-empty input → ≥1 unit" invariant. - Renamed the literal 1 to MIN_UNITS_PER_NONEMPTY_FILE constant with javadoc. - Added javadoc paragraphs on both the DocumentClassifier interface and DefaultDocumentClassifier explaining the two-floor design so the next reader doesn't conclude minChargeUnits is silently inert. #6 (LOW) Math.toIntExact on the multi-file group cap cast - Replaces (int) Math.max(1L, Math.min(groupCap, rawUnitsSum)) with Math.toIntExact(...). Theoretical (HTTP body limits make 2.15M files impossible) but matches the saturatedAdd care taken everywhere else. Single-file path got the same treatment. #8 (LOW) Document the @Version asymmetry + INTEGER-vs-BIGINT widths - WalletPolicy and WalletEntitlementSnapshot now carry javadoc explaining why no @Version: admin-only writes (wallet_policy) or full-row recomputation (entitlement snapshot) — no read-modify-write race exists. - V11 wallet_ledger section now carries a comment block explaining the width split: amount_units INTEGER (per-row delta, always small), but cap_units / period_spend_units / period_cap_units BIGINT (accumulate across a period, headroom matters). #5 (LOW) — PR description drift — handled separately via gh pr edit. #7 (LOW) — ddl-auto=update with Flyway — deferred to its own PR. Full :saas:test BUILD SUCCESSFUL with the new SaasJpaConfigScanTest passing 2/2.
Member
Author
|
Thanks for the review — all eight concerns validated and addressed in commit Fixed in this commit
Negotiated and accepted
Deferred to a separate cleanup PR
Tests + build
Knock-ons to stacked PRs
Both will be rebased onto this commit next. |
The review-response commit landed several oversized javadoc paragraphs and SQL comment blocks. Trim each to the essential what/why; drop the inline "see <design-doc-section>" / "PR-X1" markers — code shouldn't carry roadmap references. Net: ~70 fewer lines of comment, same information density on the points that matter.
Frooodle
approved these changes
May 29, 2026
jbrunton96
approved these changes
May 29, 2026
ConnorYoh
added a commit
that referenced
this pull request
May 29, 2026
PR-I1 (service half). Built off #6460 — sibling of #6464 (lineage primitives), both stacked on payg-i2-document-classifier so they can review independently. Lookup precedence: - PaygTeamExtensions.pricingPolicyId override → load that policy - else load the pricing_policy row with is_default = TRUE - if the override points at a deleted row, fall back to default (logs a warn — safety net for racing deletes) Cache: - 30s Caffeine, keyed by teamId, max 10k entries — correctness floor. - Invalidated on PolicyChangedEvent from any source (admin mutation here, or cross-instance via the Postgres LISTEN runner). - Admin reads bypass the cache (getEffectivePolicyUncached) so admins always see their own writes. Writes (service layer, transactional, fire-after-commit event): - create(draft) — rejects pre-set policy_id or is_default=true (promotion must go through setDefault so the partial unique idx is freed first). - setDefault(id) — clearDefaultFlag + flip; idempotent no-op if already default (no event fired in that case). - setTeamOverride(teamId, policyId|null) — validate policy exists before save. Admin REST surface — /api/v1/admin/payg/... - GET /policies, GET /policies/{id}, POST /policies - POST /policies/{id}/set-default - PUT /teams/{teamId}/policy-override - GET /teams/{teamId}/effective-policy (bypasses cache) All gated by @PreAuthorize("hasRole('ADMIN')"). Validation errors return 400, unknown rows return 404. LISTEN/NOTIFY runner (PolicyChangeListener): - Opens a dedicated raw JDBC connection via DriverManager (not HikariCP — Hikari would evict idle LISTEN connections) and polls getNotifications() on a daemon thread. - Reconnects with 5s backoff on SQLException; the 30s TTL is the correctness floor during outages. - Disabled by setting payg.policy.listen.enabled=false (default on). - PgJDBC moved to compileOnly on :saas; the runtime artifact is still bundled via :proprietary. V11 migration: seed the V1 default policy (25 pages/unit, 5 MiB/unit, min charge 1, file cap 1000) + step limits per JobSource (WEB/API/DESKTOP_APP 10, PIPELINE 20). Idempotent — only inserts when no default exists. Tests: 17 PricingPolicyServiceTest + 14 PricingPolicyAdminControllerTest, all passing. Coverage moves 10.87% → 12.40% LINE and 11.89% → 13.85% INSTRUCTION. Counterpart Supabase migration (NOTIFY trigger function + per-table triggers on pricing_policy*, payg_team_extensions overrides, wallet_policy; plus the default-policy seed) ships on a separate SaaS PR on payg-i1-pricing-policy-service.
ConnorYoh
added a commit
that referenced
this pull request
May 29, 2026
V11 has shipped to main via #6460; modifying it now would change its Flyway checksum and break every deployment that already ran V11. The seed (V1 default pricing policy row + per-JobSource step limits) lands in V12__seed_default_payg_policy.sql instead — purely additive, idempotent (WHERE NOT EXISTS), no schema change. Also drops a stray "PR-C5" roadmap marker that survived the trim pass.
ConnorYoh
added a commit
that referenced
this pull request
May 29, 2026
Three orthogonal interfaces, each with one production impl. Designed so the hash algorithm, the storage backend, and the matching policy can each be swapped independently without changes elsewhere. LineageSignatureExtractor — given a Path, returns Set<LineageSignature>. File-based (not stream-based) so a future PDF-aware extractor can open the same file with jpdfium / PDFBox and pull /ID[0] or a content-stream hash alongside the byte hash. Multiple extractors compose at the detector layer via Spring auto-wiring; the detector unions their results. Default impl: ByteHashSignatureExtractor — SHA-256 of file bytes via a 64 KiB-buffered DigestInputStream. Hardware-accelerated by the JVM. JobLineageStore — record(jobId, signatures, kind) / findOpenJobForSignatures(userId, candidates, window) / pruneOlderThan. Knows nothing about storage technology. Production impl JpaJobLineageStore backs onto job_artifact_hash via a single joined query. A future RedisJobLineageStore is a drop-in. HashLineageDetector — high-level API. Delegates extraction to the configured extractors and storage to the store. Reads payg.lineage.workflow-window (default PT5M) from config. Future hook: when wallet_policy.auto_group_strategy lookup lands, the detect() method short-circuits to empty when the team has opted out. Schema: content_hash widened from CHAR(64) to VARCHAR(128) so non-SHA-256 signatures fit. Storage form is "type:value" — sha256:hexdigest, pdf-id:uuid, etc. coexist on the same column. Tests: InMemoryJobLineageStore + FakeFileSignatureExtractor under src/test exercise the detector end-to-end without a database. Covers same-user, within-window, status=OPEN, multi-signature, most-recent-wins, record+detect round-trip, and extractor-failure isolation. Stacked PR: branched off payg-i2-document-classifier because the JPA store + JobArtifactHash entity depend on PR #6460's data model. Once that merges, this PR rebases to target main.
brosequist
pushed a commit
to brosequist/Stirling-PDF
that referenced
this pull request
May 29, 2026
…ols#6469) ## What this is PR-I1 service half from `notes/PAYG_DESIGN.md`. Built on top of the data model from Stirling-Tools#6460 — answers "what pricing policy applies to this team right now?" with a fast cache and an admin write surface. ## Scope | Piece | Where | |---|---| | `PricingPolicyService` — `getEffectivePolicy(teamId)` with 30s Caffeine cache + admin write paths | `app/saas/.../payg/policy/PricingPolicyService.java` | | `PolicyChangedEvent` — published after admin writes for in-process cache invalidation | `app/saas/.../payg/policy/PolicyChangedEvent.java` | | Admin REST — list / get / create / set-default / set team override / get effective | `app/saas/.../payg/policy/admin/PricingPolicyAdminController.java` + DTOs | | `PricingPolicyRepository.clearDefaultFlag()` — atomic clear for set-default | repository update | | `SaasJpaConfigScanTest` — drift guard against the JPA scan paths going stale (carried over from the Stirling-Tools#6460 review concern) | new test | | V12 default-policy seed (`v1-initial`, 25 pages/unit, 5 MiB/unit, per-`JobSource` step limits) | `V12__seed_default_payg_policy.sql` | ## Lookup precedence 1. `PaygTeamExtensions.pricingPolicyId` set → return that policy 2. Else return the `pricing_policy` row with `is_default = TRUE` 3. Override row points at a deleted policy → log warn, fall back to default (safety net for racing deletes) 4. No default → `IllegalStateException` (V12 seed guarantees one exists) ## Cache behaviour - 30s `expireAfterWrite` Caffeine, max 10k entries, keyed by `teamId`. - **Single correctness model: the TTL.** Cross-instance propagation is at-most-30-seconds. The writer instance sees its own change immediately via the `PolicyChangedEvent` after-commit publish. Other instances pick it up on the next TTL expiry. - Admin reads use `getEffectivePolicyUncached` so admins always see their own write straight back. **Why no LISTEN/NOTIFY runner.** An earlier cut of this PR included a Postgres `LISTEN policy_changed` runner so cross-instance propagation was instant. Dropped — admin policy changes are events-per-week and the 30s TTL is already the correctness floor; the listener was ~250 lines of nontrivial code (raw JDBC outside HikariCP, daemon thread, reconnect loop, lock-protected connection lifecycle) for a use case that isn't on the hot path. Trade-off is documented in `notes/PAYG_DESIGN.md` §9 with three concrete triggers that would justify reintroducing it (aggressive cap enforcement, Redis landing for other reasons, real-time admin UI). ## Writes — transactional, fire `PolicyChangedEvent` after commit - `create(draft)` — rejects pre-set `policy_id` or `is_default=true` (promotion must go through `setDefault` so the partial unique idx is freed first). - `setDefault(id)` — atomically clears the existing default via `clearDefaultFlag()` then flips the new row. Idempotent: silent no-op if the row is already default. - `setTeamOverride(teamId, policyId | null)` — validates the policy exists before save; `null` clears the override. `publishOnCommit` uses `TransactionSynchronizationManager.afterCommit` so listeners never see pre-commit state. Outside a transaction (test paths) falls through to immediate publish. ## Admin REST surface — `/api/v1/admin/payg/...` All endpoints `@PreAuthorize("hasRole('ADMIN')")`: - `GET /policies` — list all - `GET /policies/{id}` — read one - `POST /policies` — create new (non-default) - `POST /policies/{id}/set-default` — atomic promote - `PUT /teams/{teamId}/policy-override` — set or clear per-team override - `GET /teams/{teamId}/effective-policy` — cache-bypassing live read Validation errors → 400, unknown rows → 404. ## Counterpart Supabase PR [`Stirling-PDF-SaaS#298`](Stirling-Tools/Stirling-PDF-SaaS#298) — seeds the same V1 default policy on the Supabase side via `20260528000002_payg_seed_default_policy.sql`. ## Tests - 17 × `PricingPolicyServiceTest` — lookup precedence, cache hit/miss, invalidation on event, mutation paths publishing event, error cases. - 14 × `PricingPolicyAdminControllerTest` — every endpoint's happy path + error mapping, DTO defensive-copy invariant. - 2 × `SaasJpaConfigScanTest` — reflection-based guard that `payg.repository` is in `@EnableJpaRepositories` and `payg` is in `@EntityScan`. Without this, new sub-packages can silently fail to wire at runtime — same class of bug that the Stirling-Tools#6460 review caught. Full `:saas:test` BUILD SUCCESSFUL. ## Design doc `notes/PAYG_DESIGN.md` §7.4 PR-I1 — completes the service half (the schema half landed in Stirling-Tools#6460). §9 carries the 30s-TTL trade-off note.
ConnorYoh
added a commit
that referenced
this pull request
Jun 2, 2026
Three orthogonal interfaces, each with one production impl. Designed so the hash algorithm, the storage backend, and the matching policy can each be swapped independently without changes elsewhere. LineageSignatureExtractor — given a Path, returns Set<LineageSignature>. File-based (not stream-based) so a future PDF-aware extractor can open the same file with jpdfium / PDFBox and pull /ID[0] or a content-stream hash alongside the byte hash. Multiple extractors compose at the detector layer via Spring auto-wiring; the detector unions their results. Default impl: ByteHashSignatureExtractor — SHA-256 of file bytes via a 64 KiB-buffered DigestInputStream. Hardware-accelerated by the JVM. JobLineageStore — record(jobId, signatures, kind) / findOpenJobForSignatures(userId, candidates, window) / pruneOlderThan. Knows nothing about storage technology. Production impl JpaJobLineageStore backs onto job_artifact_hash via a single joined query. A future RedisJobLineageStore is a drop-in. HashLineageDetector — high-level API. Delegates extraction to the configured extractors and storage to the store. Reads payg.lineage.workflow-window (default PT5M) from config. Future hook: when wallet_policy.auto_group_strategy lookup lands, the detect() method short-circuits to empty when the team has opted out. Schema: content_hash widened from CHAR(64) to VARCHAR(128) so non-SHA-256 signatures fit. Storage form is "type:value" — sha256:hexdigest, pdf-id:uuid, etc. coexist on the same column. Tests: InMemoryJobLineageStore + FakeFileSignatureExtractor under src/test exercise the detector end-to-end without a database. Covers same-user, within-window, status=OPEN, multi-signature, most-recent-wins, record+detect round-trip, and extractor-failure isolation. Stacked PR: branched off payg-i2-document-classifier because the JPA store + JobArtifactHash entity depend on PR #6460's data model. Once that merges, this PR rebases to target main.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description of Changes
Two layers — the
DocumentClassifierutility plus the full data model for the new billing engine. Nothing wires the entities into application behaviour yet; services and controllers land in follow-up PRs.Companion PR: Stirling-PDF-SaaS#296 — Supabase migration for the v3 dev branch, schema-equivalent to the Flyway migration in this PR.
1. DocumentClassifier (under
payg.docs)DocumentClassifiercomputes the doc-unit cost of an uploaded file (or multi-file input) under aPricingPolicy. PDFs read page count viastirling.software.jpdfium.PdfDocument; non-PDFs are bytes-only. Formula:max(ceil(pages / docPagesPerUnit), ceil(bytes / docBytesPerUnit))clamped to[1, fileUnitCap]. Multi-file is the sum of raw per-file units capped atfileUnitCap × file_count.Two floors, by design: the classifier returns
docUnitswith an absolute1floor for non-empty input; the policy-levelminChargeUnitsis intentionally applied later, at process-open time inJobChargeService, per design § 3.4 (unitsForProcess = max(policy.min_charge_units, docUnits)). Documented in the interface + impl javadoc.Upload bytes are materialised through
TempFileManager.createManagedTempFileso jpdfium gets aPath; the temp file auto-deletes on close.Twelve tests, all in-memory fixtures generated with PDFBox at test time — no committed binary blobs.
2. PAYG data model (under
payg.*)JPA entities, repositories, and a Flyway migration covering the full schema in §6 of the design.
Enums (
payg.model):JobSource,ProcessType,JobStatus,JobStepStatus,ArtifactKind,LedgerEntryType,LedgerBucket,ReferenceType,EntitlementState,FeatureSet,FeatureGate,WalletEngine,CapPeriod,AutoGroupStrategy.Entities + repositories:
PricingPolicypricing_policystepLimitsisMap<JobSource, Integer>persisted via normalised child tablepricing_policy_step_limit.stripePriceIdsisSet<String>persisted viapricing_policy_stripe_price— currency comes fromstripe.pricesvia Sync Engine, not stored locally.ProcessingJobprocessing_jobstep_countandlast_step_at.ProcessingJobStepprocessing_job_stepJobArtifactHashjob_artifact_hash(job_id, content_hash, kind).content_hash VARCHAR(128)so multiple signature schemes coexist as"type:value"storage keys. Lineage detector queries this.WalletLedgerEntrywallet_ledgeramount_units. Two unique indexes kill double-posting.WalletPolicywallet_policy@Version— admin-only writes (documented in javadoc).WalletEntitlementSnapshotwallet_entitlement_snapshot(team_id, user_id);user_id = 0is the team-wide sentinel. No@Version— full-row recompute viaEntitlementService.recompute(documented in javadoc).PaygShadowChargepayg_shadow_chargePAYG_SHADOWengine mode.PaygTeamExtensionspayg_team_extensionsteamscarryingpricing_policy_id(per-team override) +stripe_customer_id. Sidecar pattern (mirrorssaas_team_extensions) so OSS Hibernate ddl-auto never sees PAYG columns onteams.Column adds:
team_memberships.cap_units(optional per-member sub-cap)Width split (intentional, documented in V11): per-row deltas (
wallet_ledger.amount_units,processing_job.charged_units) areINTEGERbecause no single charge realistically approaches 2B units. Cap and period-rollup columns (team_memberships.cap_units,wallet_policy.cap_units,wallet_entitlement_snapshot.period_spend_units / period_cap_units) areBIGINTbecause they accumulate across a billing period and admins may legitimately set headroom-cap values into the millions.JPA wiring:
SaasJpaConfigwas updated to includestirling.software.saas.payg.repositoryin@EnableJpaRepositories.basePackagesandstirling.software.saas.paygin@EntityScan(coverspayg.policy/payg.job/payg.wallet/payg.entitlement/payg.shadowrecursively). NewSaasJpaConfigScanTestreads the annotations reflectively and asserts every expected package is wired — catches the next time someone adds a new sub-package without updating the scan paths.Migration:
V11__saas_payg_model.sql(purely additive). Schema-equivalent to the Supabase migration in the companion PR — including theVARCHAR(128) content_hashwidth that's needed for the multi-signature-scheme storage encoding the lineage layer uses.3. Smoke tests
PaygEntitiesSmokeTestexercises each entity via the no-arg ctor JPA requires, plus getter/setter round-trips and composite-key equality — catches Lombok/annotation regressions without needing a database. Real-DB integration coverage lands alongside the services that consume each entity.Why this is safe to land now
Open decisions made
JobSourcerather than byProcessType. Captures the "self-hosted gets a different knob" framing in earlier feedback. Trivially overridable per pricing policy version.pricing_policy(per Connor's review on add fileInput widget to multiSelect #296). Typed columns, queryable directly, no JSON parsing.pricing_policy_stripe_price— it lives onstripe.prices.currencyand is resolved via Sync Engine. App is currency-blind.Rollback
Straight
git reverton this PR. The Supabase migration in #296 is additive and can be left in place safely — the running app ignores tables it doesn't reference.Checklist
task check(via./gradlew :saas:testwithENABLE_SAAS=true) — passes