Skip to content

Commit 83ea07e

Browse files
authored
saas: DocumentClassifier + PAYG data model (Stirling-Tools#6460)
# Description of Changes Two layers — the `DocumentClassifier` utility plus the full data model for the new billing engine. Nothing wires the entities into application behaviour yet; services and controllers land in follow-up PRs. **Companion PR:** [Stirling-PDF-SaaS#296](Stirling-Tools/Stirling-PDF-SaaS#296) — Supabase migration for the v3 dev branch, schema-equivalent to the Flyway migration in this PR. ## 1. DocumentClassifier (under `payg.docs`) `DocumentClassifier` computes the doc-unit cost of an uploaded file (or multi-file input) under a `PricingPolicy`. PDFs read page count via `stirling.software.jpdfium.PdfDocument`; non-PDFs are bytes-only. Formula: `max(ceil(pages / docPagesPerUnit), ceil(bytes / docBytesPerUnit))` clamped to `[1, fileUnitCap]`. Multi-file is the sum of raw per-file units capped at `fileUnitCap × file_count`. Two floors, by design: the classifier returns `docUnits` with an absolute `1` floor for non-empty input; the policy-level `minChargeUnits` is intentionally applied later, at process-open time in `JobChargeService`, per design § 3.4 (`unitsForProcess = max(policy.min_charge_units, docUnits)`). Documented in the interface + impl javadoc. Upload bytes are materialised through `TempFileManager.createManagedTempFile` so jpdfium gets a `Path`; the temp file auto-deletes on close. Twelve tests, all in-memory fixtures generated with PDFBox at test time — no committed binary blobs. ## 2. PAYG data model (under `payg.*`) JPA entities, repositories, and a Flyway migration covering the full schema in §6 of the design. **Enums** (`payg.model`): `JobSource`, `ProcessType`, `JobStatus`, `JobStepStatus`, `ArtifactKind`, `LedgerEntryType`, `LedgerBucket`, `ReferenceType`, `EntitlementState`, `FeatureSet`, `FeatureGate`, `WalletEngine`, `CapPeriod`, `AutoGroupStrategy`. **Entities + repositories:** | Entity | Table | Notes | |---|---|---| | `PricingPolicy` | `pricing_policy` | Promoted from a record. `stepLimits` is `Map<JobSource, Integer>` persisted via normalised child table `pricing_policy_step_limit`. `stripePriceIds` is `Set<String>` persisted via `pricing_policy_stripe_price` — currency comes from `stripe.prices` via Sync Engine, not stored locally. | | `ProcessingJob` | `processing_job` | UUID PK. Tracks lineage window via `step_count` and `last_step_at`. | | `ProcessingJobStep` | `processing_job_step` | Per-tool-call audit. | | `JobArtifactHash` | `job_artifact_hash` | Composite key `(job_id, content_hash, kind)`. `content_hash VARCHAR(128)` so multiple signature schemes coexist as `"type:value"` storage keys. Lineage detector queries this. | | `WalletLedgerEntry` | `wallet_ledger` | Append-only, signed `amount_units`. Two unique indexes kill double-posting. | | `WalletPolicy` | `wallet_policy` | Per-team engine + cap + degradation rules + lineage strategy. No `@Version` — admin-only writes (documented in javadoc). | | `WalletEntitlementSnapshot` | `wallet_entitlement_snapshot` | Composite key `(team_id, user_id)`; `user_id = 0` is the team-wide sentinel. No `@Version` — full-row recompute via `EntitlementService.recompute` (documented in javadoc). | | `PaygShadowCharge` | `payg_shadow_charge` | Per-job diff while in `PAYG_SHADOW` engine mode. | | `PaygTeamExtensions` | `payg_team_extensions` | Sidecar 1:1 with `teams` carrying `pricing_policy_id` (per-team override) + `stripe_customer_id`. Sidecar pattern (mirrors `saas_team_extensions`) so OSS Hibernate ddl-auto never sees PAYG columns on `teams`. | **Column adds:** - `team_memberships.cap_units` (optional per-member sub-cap) **Width split (intentional, documented in V11):** per-row deltas (`wallet_ledger.amount_units`, `processing_job.charged_units`) are `INTEGER` because no single charge realistically approaches 2B units. Cap and period-rollup columns (`team_memberships.cap_units`, `wallet_policy.cap_units`, `wallet_entitlement_snapshot.period_spend_units / period_cap_units`) are `BIGINT` because they accumulate across a billing period and admins may legitimately set headroom-cap values into the millions. **JPA wiring:** `SaasJpaConfig` was updated to include `stirling.software.saas.payg.repository` in `@EnableJpaRepositories.basePackages` and `stirling.software.saas.payg` in `@EntityScan` (covers `payg.policy` / `payg.job` / `payg.wallet` / `payg.entitlement` / `payg.shadow` recursively). New `SaasJpaConfigScanTest` reads the annotations reflectively and asserts every expected package is wired — catches the next time someone adds a new sub-package without updating the scan paths. **Migration:** `V11__saas_payg_model.sql` (purely additive). Schema-equivalent to the Supabase migration in the companion PR — including the `VARCHAR(128) content_hash` width that's needed for the multi-signature-scheme storage encoding the lineage layer uses. ## 3. Smoke tests `PaygEntitiesSmokeTest` exercises each entity via the no-arg ctor JPA requires, plus getter/setter round-trips and composite-key equality — catches Lombok/annotation regressions without needing a database. Real-DB integration coverage lands alongside the services that consume each entity. ## Why this is safe to land now - All schema changes are additive — no existing rows modified, no columns dropped. - The entities are not yet referenced from any production code path; they exist for the next PRs to build on. - The v3 Supabase dev branch picks up the schema via the companion PR; the main repo's Flyway migration applies the same shape when an instance boots against a freshly-migrated v3 database. ## Open decisions made - **Step-limits keyed by `JobSource`** rather than by `ProcessType`. Captures the "self-hosted gets a different knob" framing in earlier feedback. Trivially overridable per pricing policy version. - **Step limits + Stripe price IDs normalised into child tables** rather than JSONB on `pricing_policy` (per Connor's review on #296). Typed columns, queryable directly, no JSON parsing. - **Currency dropped from `pricing_policy_stripe_price`** — it lives on `stripe.prices.currency` and is resolved via Sync Engine. App is currency-blind. ## Rollback Straight `git revert` on this PR. The Supabase migration in #296 is additive and can be left in place safely — the running app ignores tables it doesn't reference. --- ## Checklist - [x] I have read the [Contribution Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md) - [x] I have read the [Stirling-PDF Developer Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md) - [x] I have performed a self-review of my own code - [x] My changes generate no new warnings - [x] I have run `task check` (via `./gradlew :saas:test` with `ENABLE_SAAS=true`) — passes
1 parent 61ebe97 commit 83ea07e

41 files changed

Lines changed: 2069 additions & 3 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

app/saas/src/main/java/stirling/software/saas/config/SaasJpaConfig.java

Lines changed: 9 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -5,18 +5,24 @@
55
import org.springframework.context.annotation.Profile;
66
import org.springframework.data.jpa.repository.config.EnableJpaRepositories;
77

8-
/** Registers the {@code :saas} module's entities and repositories with Spring Data JPA. */
8+
/**
9+
* Registers the {@code :saas} module's entities and repositories with Spring Data JPA. Any new
10+
* package holding {@code @Repository} or {@code @Entity} classes must be added here, or the beans
11+
* won't wire at startup.
12+
*/
913
@Configuration
1014
@Profile("saas")
1115
@EnableJpaRepositories(
1216
basePackages = {
1317
"stirling.software.saas.repository",
1418
"stirling.software.saas.billing.repository",
15-
"stirling.software.saas.ai.repository"
19+
"stirling.software.saas.ai.repository",
20+
"stirling.software.saas.payg.repository"
1621
})
1722
@EntityScan({
1823
"stirling.software.saas.model",
1924
"stirling.software.saas.billing.model",
20-
"stirling.software.saas.ai.model"
25+
"stirling.software.saas.ai.model",
26+
"stirling.software.saas.payg"
2127
})
2228
public class SaasJpaConfig {}

app/saas/src/main/java/stirling/software/saas/model/TeamMembership.java

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -73,6 +73,13 @@ public class TeamMembership implements Serializable {
7373
@Column(name = "updated_at", nullable = false)
7474
private LocalDateTime updatedAt;
7575

76+
/**
77+
* Optional per-member spend cap inside the team's wallet, in doc units. NULL means the member
78+
* is bounded only by the team-wide cap.
79+
*/
80+
@Column(name = "cap_units")
81+
private Long capUnits;
82+
7683
public boolean isLeader() {
7784
return role == TeamRole.LEADER;
7885
}
Lines changed: 173 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,173 @@
1+
package stirling.software.saas.payg.docs;
2+
3+
import java.io.IOException;
4+
import java.io.InputStream;
5+
import java.io.OutputStream;
6+
import java.nio.file.Files;
7+
import java.util.List;
8+
import java.util.Objects;
9+
10+
import org.springframework.context.annotation.Profile;
11+
import org.springframework.stereotype.Component;
12+
import org.springframework.web.multipart.MultipartFile;
13+
14+
import lombok.RequiredArgsConstructor;
15+
import lombok.extern.slf4j.Slf4j;
16+
17+
import stirling.software.common.util.TempFile;
18+
import stirling.software.common.util.TempFileManager;
19+
import stirling.software.jpdfium.PdfDocument;
20+
import stirling.software.saas.payg.policy.PricingPolicy;
21+
22+
/**
23+
* Reads pages via jpdfium for PDF inputs; treats every other content type as bytes-only.
24+
*
25+
* <p>For PDFs, units are the larger of {@code ceil(pages / docPagesPerUnit)} and {@code ceil(bytes
26+
* / docBytesPerUnit)}. For non-PDFs, only the bytes axis contributes. A single file is clamped to
27+
* {@code [1, policy.fileUnitCap]}; a multi-file group is clamped to {@code [1, policy.fileUnitCap *
28+
* file_count]} applied to the sum of raw per-file units. Malformed/encrypted PDFs fall back to
29+
* bytes-only.
30+
*
31+
* <p>{@code policy.minChargeUnits} is applied by the charge service, not here. The classifier only
32+
* enforces an absolute floor of {@link #MIN_UNITS_PER_NONEMPTY_FILE} so callers can rely on
33+
* "non-empty input → at least 1 unit".
34+
*/
35+
@Slf4j
36+
@Component
37+
@Profile("saas")
38+
@RequiredArgsConstructor
39+
public class DefaultDocumentClassifier implements DocumentClassifier {
40+
41+
private static final String PDF_CONTENT_TYPE = "application/pdf";
42+
private static final String DEFAULT_CONTENT_TYPE = "application/octet-stream";
43+
44+
/** Floor for non-empty input. Distinct from {@code policy.minChargeUnits} (applied later). */
45+
private static final int MIN_UNITS_PER_NONEMPTY_FILE = 1;
46+
47+
private final TempFileManager tempFileManager;
48+
49+
@Override
50+
public DocumentMetrics classify(MultipartFile file, PricingPolicy policy) {
51+
Objects.requireNonNull(file, "file");
52+
Objects.requireNonNull(policy, "policy");
53+
54+
FileFacts facts = inspect(file);
55+
long rawUnits = computeRawUnits(facts.pages, facts.bytes, policy);
56+
// toIntExact: fail loud on overflow rather than silently wrapping a billing number.
57+
int units =
58+
Math.toIntExact(
59+
Math.max(
60+
MIN_UNITS_PER_NONEMPTY_FILE,
61+
Math.min(policy.getFileUnitCap(), rawUnits)));
62+
return new DocumentMetrics(facts.pages, facts.bytes, facts.contentType, units);
63+
}
64+
65+
@Override
66+
public DocumentMetrics classify(List<MultipartFile> files, PricingPolicy policy) {
67+
Objects.requireNonNull(files, "files");
68+
Objects.requireNonNull(policy, "policy");
69+
if (files.isEmpty()) {
70+
throw new IllegalArgumentException("files must not be empty");
71+
}
72+
73+
int totalPages = 0;
74+
long totalBytes = 0;
75+
long rawUnitsSum = 0;
76+
String firstContentType = null;
77+
78+
for (MultipartFile file : files) {
79+
FileFacts facts = inspect(file);
80+
// Sum the *raw* (unclamped) per-file units so the group cap below can actually bind.
81+
// Per-file clamping in this loop would make the group cap a no-op.
82+
rawUnitsSum =
83+
saturatedAdd(rawUnitsSum, computeRawUnits(facts.pages, facts.bytes, policy));
84+
totalPages = saturatedAdd(totalPages, facts.pages);
85+
totalBytes = saturatedAdd(totalBytes, facts.bytes);
86+
if (firstContentType == null) {
87+
firstContentType = facts.contentType;
88+
}
89+
}
90+
91+
long groupCap = (long) policy.getFileUnitCap() * files.size();
92+
// toIntExact: fail loud on overflow rather than silently wrapping.
93+
int totalUnits =
94+
Math.toIntExact(
95+
Math.max(
96+
(long) MIN_UNITS_PER_NONEMPTY_FILE,
97+
Math.min(groupCap, rawUnitsSum)));
98+
99+
return new DocumentMetrics(
100+
totalPages,
101+
totalBytes,
102+
firstContentType != null ? firstContentType : DEFAULT_CONTENT_TYPE,
103+
totalUnits);
104+
}
105+
106+
private FileFacts inspect(MultipartFile file) {
107+
long bytes = file.getSize();
108+
String contentType =
109+
file.getContentType() != null ? file.getContentType() : DEFAULT_CONTENT_TYPE;
110+
int pages = isPdf(contentType, file.getOriginalFilename()) ? readPageCount(file) : 0;
111+
return new FileFacts(pages, bytes, contentType);
112+
}
113+
114+
private static long computeRawUnits(int pages, long bytes, PricingPolicy policy) {
115+
long pageUnits = pages > 0 ? ceilDiv(pages, policy.getDocPagesPerUnit()) : 0L;
116+
long byteUnits = ceilDiv(bytes, policy.getDocBytesPerUnit());
117+
return Math.max(pageUnits, byteUnits);
118+
}
119+
120+
private static long ceilDiv(long numerator, long divisor) {
121+
if (numerator <= 0) {
122+
return 0;
123+
}
124+
return (numerator + divisor - 1) / divisor;
125+
}
126+
127+
private static boolean isPdf(String contentType, String filename) {
128+
if (PDF_CONTENT_TYPE.equalsIgnoreCase(contentType)) {
129+
return true;
130+
}
131+
return filename != null && filename.toLowerCase().endsWith(".pdf");
132+
}
133+
134+
/**
135+
* Materialises the upload to a managed temp file and asks jpdfium for the page count. Returns 0
136+
* if the file can't be parsed — the byte-derived axis still produces a charge.
137+
*/
138+
private int readPageCount(MultipartFile file) {
139+
try (TempFile temp = tempFileManager.createManagedTempFile(".pdf")) {
140+
try (InputStream in = file.getInputStream();
141+
OutputStream out = Files.newOutputStream(temp.getPath())) {
142+
in.transferTo(out);
143+
}
144+
try (PdfDocument doc = PdfDocument.open(temp.getPath())) {
145+
return doc.pageCount();
146+
}
147+
} catch (IOException | RuntimeException e) {
148+
log.debug(
149+
"Could not read PDF page count for {} ({}); falling back to bytes-only units",
150+
file.getOriginalFilename(),
151+
e.getClass().getSimpleName());
152+
return 0;
153+
}
154+
}
155+
156+
private static int saturatedAdd(int a, int b) {
157+
long sum = (long) a + b;
158+
if (sum > Integer.MAX_VALUE) {
159+
return Integer.MAX_VALUE;
160+
}
161+
return (int) sum;
162+
}
163+
164+
private static long saturatedAdd(long a, long b) {
165+
try {
166+
return Math.addExact(a, b);
167+
} catch (ArithmeticException e) {
168+
return Long.MAX_VALUE;
169+
}
170+
}
171+
172+
private record FileFacts(int pages, long bytes, String contentType) {}
173+
}
Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,25 @@
1+
package stirling.software.saas.payg.docs;
2+
3+
import java.util.List;
4+
5+
import org.springframework.web.multipart.MultipartFile;
6+
7+
import stirling.software.saas.payg.policy.PricingPolicy;
8+
9+
/**
10+
* Computes the doc-unit cost of an uploaded file (or multi-file input) under a given policy.
11+
*
12+
* <p>Returns {@code docUnits} with an absolute floor of 1 for non-empty input. {@code
13+
* policy.minChargeUnits} is applied at charge time, not here.
14+
*/
15+
public interface DocumentClassifier {
16+
17+
/** Classify a single uploaded file. Returns at least 1 unit, capped at {@code fileUnitCap}. */
18+
DocumentMetrics classify(MultipartFile file, PricingPolicy policy);
19+
20+
/**
21+
* Classify a multi-file input (e.g. a merge or overlay). Returns the sum of each file's raw
22+
* units, capped at {@code fileUnitCap × files.size()} and floored at 1.
23+
*/
24+
DocumentMetrics classify(List<MultipartFile> files, PricingPolicy policy);
25+
}
Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
1+
package stirling.software.saas.payg.docs;
2+
3+
/**
4+
* Output of {@link DocumentClassifier#classify}. {@code pages} is {@code 0} for non-PDF inputs.
5+
*
6+
* @param pages page count (0 for non-PDFs and for files whose page count couldn't be read)
7+
* @param bytes raw byte length of the file
8+
* @param contentType MIME type as reported by the upload, or {@code "application/octet-stream"}
9+
* when unknown
10+
* @param docUnits computed unit cost, clamped to the policy's {@code fileUnitCap}
11+
*/
12+
public record DocumentMetrics(int pages, long bytes, String contentType, int docUnits) {}
Lines changed: 112 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,112 @@
1+
package stirling.software.saas.payg.entitlement;
2+
3+
import java.io.Serializable;
4+
import java.time.LocalDateTime;
5+
import java.util.ArrayList;
6+
import java.util.List;
7+
import java.util.Objects;
8+
9+
import org.hibernate.annotations.CreationTimestamp;
10+
import org.hibernate.annotations.JdbcTypeCode;
11+
import org.hibernate.type.SqlTypes;
12+
13+
import jakarta.persistence.Column;
14+
import jakarta.persistence.Embeddable;
15+
import jakarta.persistence.EmbeddedId;
16+
import jakarta.persistence.Entity;
17+
import jakarta.persistence.EnumType;
18+
import jakarta.persistence.Enumerated;
19+
import jakarta.persistence.Table;
20+
21+
import lombok.Getter;
22+
import lombok.NoArgsConstructor;
23+
import lombok.Setter;
24+
25+
import stirling.software.saas.payg.model.EntitlementState;
26+
import stirling.software.saas.payg.model.FeatureGate;
27+
import stirling.software.saas.payg.model.FeatureSet;
28+
29+
/**
30+
* Cached entitlement state for the team (one row with {@code user_id = 0}, the team-wide sentinel)
31+
* plus optional per-member rows when a member sub-cap is configured. Read on the hot path by the
32+
* entitlement guard.
33+
*
34+
* <p>Composite PK {@code (team_id, user_id)} uses 0 as the team-wide sentinel because Postgres
35+
* treats {@code NULL} as not-equal-to-NULL in unique constraints — 0 keeps the PK well-defined.
36+
*
37+
* <p>No {@code @Version} — rows are produced by full-row recompute, no read-modify-write race.
38+
*/
39+
@Entity
40+
@Table(name = "wallet_entitlement_snapshot")
41+
@NoArgsConstructor
42+
@Getter
43+
@Setter
44+
public class WalletEntitlementSnapshot implements Serializable {
45+
46+
private static final long serialVersionUID = 1L;
47+
48+
public static final long TEAM_WIDE_USER_ID = 0L;
49+
50+
@EmbeddedId private WalletEntitlementSnapshotId id;
51+
52+
@Column(name = "period_start", nullable = false)
53+
private LocalDateTime periodStart;
54+
55+
@Column(name = "period_end", nullable = false)
56+
private LocalDateTime periodEnd;
57+
58+
@Column(name = "period_spend_units", nullable = false)
59+
private Long periodSpendUnits = 0L;
60+
61+
@Column(name = "period_cap_units")
62+
private Long periodCapUnits;
63+
64+
@Enumerated(EnumType.STRING)
65+
@Column(name = "state", nullable = false, length = 16)
66+
private EntitlementState state = EntitlementState.FULL;
67+
68+
@Enumerated(EnumType.STRING)
69+
@Column(name = "feature_set", nullable = false, length = 32)
70+
private FeatureSet featureSet = FeatureSet.FULL;
71+
72+
@JdbcTypeCode(SqlTypes.JSON)
73+
@Column(name = "enabled_gates", columnDefinition = "jsonb", nullable = false)
74+
private List<FeatureGate> enabledGates = new ArrayList<>();
75+
76+
@CreationTimestamp
77+
@Column(name = "computed_at", nullable = false, updatable = false)
78+
private LocalDateTime computedAt;
79+
80+
@Embeddable
81+
@NoArgsConstructor
82+
@Getter
83+
@Setter
84+
public static class WalletEntitlementSnapshotId implements Serializable {
85+
86+
private static final long serialVersionUID = 1L;
87+
88+
@Column(name = "team_id", nullable = false)
89+
private Long teamId;
90+
91+
/** Use {@link #TEAM_WIDE_USER_ID} for the team-wide row. */
92+
@Column(name = "user_id", nullable = false)
93+
private Long userId;
94+
95+
public WalletEntitlementSnapshotId(Long teamId, Long userId) {
96+
this.teamId = teamId;
97+
this.userId = userId;
98+
}
99+
100+
@Override
101+
public boolean equals(Object o) {
102+
if (this == o) return true;
103+
if (!(o instanceof WalletEntitlementSnapshotId other)) return false;
104+
return Objects.equals(teamId, other.teamId) && Objects.equals(userId, other.userId);
105+
}
106+
107+
@Override
108+
public int hashCode() {
109+
return Objects.hash(teamId, userId);
110+
}
111+
}
112+
}

0 commit comments

Comments
 (0)