Commit e0e739c
authored
TML-3061: PostgreSQL and SQLite codec descriptor protocols (#1051)
## Linked issue
Refs
[TML-3061](https://linear.app/prisma-company/issue/TML-3061/target-codec-descriptor-foundations)
Base: `main`. Predecessor
[#1023](#1023) is merged, so
this PR has no stacked-base prerequisite.
## At a glance
```ts
nativeTypeFor(ref: CodecRef): string {
return this.nativeType(this.validateParams(ref));
}
projectJson(expression: ProjectionExpr, ref: CodecRef): ProjectionExpr {
const params = this.validateParams(ref);
return ref.many === true
? this.jsonArrayProjection(expression, params)
: this.jsonProjection(expression, params);
}
```
A PostgreSQL descriptor now accepts the generic `CodecRef` that survives
framework composition, validates its erased parameters, and enters
target-owned typed behavior only after validation.
## Decision
This PR ships the public PostgreSQL and SQLite codec descriptor
protocols that the lossless JSON projection work will execute through:
explicit generic-to-target adapters, narrow authoring helpers,
structurally validated immutable registries, and coherent
runtime/control adapter composition. It migrates every affected built-in
and the pgvector, PostGIS, and arktype-json extensions to those
protocols, moves PostgreSQL native-type lookup behind the validated
descriptor boundary, and adds the `PublicCodecTypes`
declaration-portability boundary plus extension-author migration
guidance.
## Reviewer notes
- The JSON projection hooks are public protocol capabilities, but
production PostgreSQL and SQLite renderers deliberately remain
pass-through. TML-3063 will activate descriptor dispatch and own the
observable lossless JSON hard cut.
- Codec JSON methods, emitted contracts, generated declarations,
fixtures, codec IDs, and current SQL/JSON representations are unchanged.
Drift in any of those surfaces was treated as a stop condition.
- `PublicCodecTypes` intentionally exposes only each codec's `input`,
`output`, and `traits` through composed contract declarations. This
keeps downstream semantic typing exact without making exported
declarations name concrete target descriptor implementation members.
- PostgreSQL extension authors must adopt the target protocol and add
the target package as a runtime dependency; direct adapter injection
also moves to target-typed `codecDescriptors`.
- Deterministic target, adapter, extension, declaration, workspace,
documentation, skill, manifest, dependency, cast, throw, upgrade,
fixture, and audit gates passed. The broad package, integration, and e2e
suites were also run, but Postgres/resource-related flakes remained
after varying and focused reruns, so their final broad confirmation is
CI-deferred and is not claimed green here.
## How it fits together
1.
[`PostgresCodecDescriptor`](packages/3-targets/3-targets/postgres/src/core/codec-descriptor.ts)
and
[`SqliteCodecDescriptor`](packages/3-targets/3-targets/sqlite/src/core/codec-descriptor.ts)
add stable structural discriminants and public template methods. Each
template method validates erased `CodecRef.typeParams` through the
descriptor's Standard Schema before calling a strongly typed target
hook.
2. `postgresCodec(...)`, `sqliteCodec(...)`,
`definePostgresCodecs(...)`, and `defineSqliteCodecs(...)` make target
ownership explicit while preserving generic SQL descriptor IDs, traits,
parameter schemas, factories, renderers, metadata, and literal types.
PostgreSQL supplies native-type plus scalar/array projection
capabilities; SQLite supplies scalar projection and reports unsupported
stored scalar-array refs.
3. Target, adapter, and ordered extension contributions are collected
once per construction plane. The
[PostgreSQL](packages/3-targets/6-adapters/postgres/src/core/codec-lookup.ts)
and
[SQLite](packages/3-targets/6-adapters/sqlite/src/core/codec-lookup.ts)
builders structurally validate that set, reject malformed, wrong-target,
raw, or duplicate descriptors, and derive ordinary materialization plus
target lookup from the same immutable registry.
4. PostgreSQL parameter rendering now resolves trusted native type names
through `nativeTypeFor(ref)` on the validated descriptor. Runtime,
control, custom, enum, extension, and array cast spellings remain pinned
to their existing SQL.
5.
[`PublicCodecTypes`](packages/2-sql/2-authoring/contract-ts/src/contract-types.ts)
projects extension codec maps to their public semantic fields before
they enter contract types. Consumer declarations retain exact
application types and traits while staying portable across package
boundaries.
6. The [codec authoring guide](docs/reference/codec-authoring-guide.md)
and the upgrade entry under
[`skills/extension-author/prisma-next-extension-upgrade/upgrades/0.16-to-0.17/`](skills/extension-author/prisma-next-extension-upgrade/upgrades/0.16-to-0.17/)
document target-owned authoring, coherent contribution sets,
parameter-ref audits, and the dormant-hook transition.
## Behavior changes & evidence
- **Extension authors can define and contribute target-owned PostgreSQL
and SQLite codecs through public APIs.** The protocols validate erased
refs before typed behavior, preserve concrete tuple/factory typing, and
use structural validation across separately loaded packages.
Implementation: [PostgreSQL descriptor
protocol](packages/3-targets/3-targets/postgres/src/core/codec-descriptor.ts),
[SQLite descriptor
protocol](packages/3-targets/3-targets/sqlite/src/core/codec-descriptor.ts).
Evidence: [PostgreSQL runtime
tests](packages/3-targets/3-targets/postgres/test/postgres-codec-descriptor.test.ts),
[SQLite runtime
tests](packages/3-targets/3-targets/sqlite/test/sqlite-codec-descriptor.test.ts).
- **PostgreSQL arrays have a target-owned default projection shape,
while SQLite rejects undefined stored scalar-array semantics.**
PostgreSQL's default lift binds the source once and preserves null
arrays, empty arrays, null elements, and ordinality; SQLite raises a
structured error for `CodecRef.many`. Implementation: [PostgreSQL
descriptor
protocol](packages/3-targets/3-targets/postgres/src/core/codec-descriptor.ts),
[SQLite descriptor
protocol](packages/3-targets/3-targets/sqlite/src/core/codec-descriptor.ts).
Evidence: [PostgreSQL descriptor
tests](packages/3-targets/3-targets/postgres/test/postgres-codec-descriptor.test.ts),
[SQLite descriptor
tests](packages/3-targets/3-targets/sqlite/test/sqlite-codec-descriptor.test.ts).
- **Runtime, control, and direct adapter paths now see one coherent
descriptor set and fail invalid composition early.** Bare adapters
remain built-ins-only; stack composition includes ordered target,
adapter, and extension contributions. Implementation: [PostgreSQL
registry
assembly](packages/3-targets/6-adapters/postgres/src/core/codec-lookup.ts),
[SQLite registry
assembly](packages/3-targets/6-adapters/sqlite/src/core/codec-lookup.ts).
Evidence: [PostgreSQL composition
tests](packages/3-targets/6-adapters/postgres/test/postgres-codec-registry-composition.test.ts),
[SQLite composition
tests](packages/3-targets/6-adapters/sqlite/test/sqlite-codec-registry-composition.test.ts).
- **First-party PostgreSQL extensions use one canonical target-typed
contribution set.** pgvector, PostGIS, and arktype-json expose the same
descriptor arrays through runtime and control metadata while retaining
current factories, application types, vector text, HEXEWKB, and
structured JSON representations. Implementation: [pgvector
codecs](packages/3-extensions/pgvector/src/core/codecs.ts), [PostGIS
codecs](packages/3-extensions/postgis/src/core/codecs.ts), [arktype-json
codec](packages/3-extensions/arktype-json/src/core/arktype-json-codec.ts).
Evidence: [pgvector adoption
tests](packages/3-extensions/pgvector/test/postgres-codec-descriptor-adoption.test.ts),
[PostGIS adoption
tests](packages/3-extensions/postgis/test/postgres-codec-descriptor-adoption.test.ts).
- **Consumer-exported contracts remain declaration-portable.** Extension
codec maps retain exact public input/output/trait semantics without
exposing target descriptor implementation structure. Implementation:
[`PublicCodecTypes`](packages/2-sql/2-authoring/contract-ts/src/contract-types.ts).
Evidence: [declaration
fixture](packages/3-extensions/pgvector/test/contract-declaration-portability.fixture.ts),
[declaration type
test](packages/3-extensions/pgvector/test/contract-declaration-portability.test-d.ts).
- **Current rendering remains unchanged while the new hooks stay
dormant.** PostgreSQL and SQLite still render codec, native, and
document projections as their wrapped expressions; composition tests
assert exact existing JSON SQL and zero descriptor projection calls.
Implementation: [PostgreSQL JSON
visitor](packages/3-targets/6-adapters/postgres/src/core/sql-renderer.ts),
[SQLite JSON
visitor](packages/3-targets/6-adapters/sqlite/src/core/adapter.ts).
Evidence: [PostgreSQL composition
tests](packages/3-targets/6-adapters/postgres/test/postgres-codec-registry-composition.test.ts),
[SQLite composition
tests](packages/3-targets/6-adapters/sqlite/test/sqlite-codec-registry-composition.test.ts).
## Compatibility / migration / risk
- This is a pre-1.0 public SPI change for PostgreSQL-bound extension
codecs. Extension descriptors must extend `PostgresCodecDescriptor` or
use `postgresCodec(...)`, define target behavior, contribute a
`definePostgresCodecs(...)` set through runtime and control paths, and
declare `@prisma-next/target-postgres` as a production dependency.
- PostgreSQL direct adapter options now accept target-typed
`codecDescriptors` in place of an independently assembled generic
`codecLookup`; SQLite gains the equivalent target-typed option.
- Invalid dynamic composition now fails during adapter construction. Raw
generic, wrong-target, structurally malformed, and duplicate descriptors
no longer survive until lookup or query lowering.
- `PublicCodecTypes` narrows incidental descriptor-derived members out
of exported contract types while preserving `input`, `output`, and
`traits`. Consumers depending on incidental implementation members
through contract types must move to the descriptor API.
- There is no data, emitted-contract, codec-ID, codec JSON, or fixture
migration in this PR. Production JSON projection remains pass-through
until TML-3063.
## Testing performed
Passed on the final branch state:
- Deterministic build, runtime test, typecheck, and lint gates for
PostgreSQL/SQLite targets, PostgreSQL/SQLite adapters, pgvector,
PostGIS, and arktype-json.
- Root `pnpm build` and `pnpm typecheck`.
- Runtime/type coverage for descriptor validation, target tuple
preservation, adapter composition, native-type/cast parity, dormant JSON
hooks, extension adoption, and `PublicCodecTypes` declaration
portability.
- Documentation, skill, and package-manifest validation; `pnpm
lint:deps`; `pnpm lint:casts`; `pnpm lint:throws`; `pnpm
check:upgrade-coverage --mode pr`; `pnpm fixtures:check`; and the
bounded closing audits.
Run, with final broad confirmation deferred to CI:
- `pnpm test:packages`
- `pnpm test:integration`
- `pnpm test:e2e`
Postgres/resource-related flakes remained after reruns at varying and
focused scopes. These three broad suites are recorded as executed, not
as passing.
## Skill update
Updated the codec authoring documentation and the extension-author
upgrade instructions in
[`skills/extension-author/prisma-next-extension-upgrade/upgrades/0.16-to-0.17/`](skills/extension-author/prisma-next-extension-upgrade/upgrades/0.16-to-0.17/).
The upgrade entry detects generic PostgreSQL extension descriptors and
walks authors through runtime dependency placement, target subclassing
or adaptation, canonical runtime/control contribution sets, required
`CodecRef.typeParams`, and behavior-preservation checks.
## Follow-ups
- TML-3063 will activate descriptor JSON projection dispatch, land the
lossless PostgreSQL/SQLite and extension representations, and remove
transitional generic metadata after production consumers have moved.
## Alternatives considered
- **Keep target behavior in generic `CodecMeta` or add a framework-owned
target map.** Rejected because target SQL/JSON behavior is open-world
and belongs to each target package; generic framework composition
remains target-neutral.
- **Validate descriptors with `instanceof`.** Rejected because
extensions may load a separate copy of a target package. Stable
structural discriminants and required methods preserve interoperability
across module identities.
- **Maintain separate ordinary and target registries.** Rejected because
the two views could disagree about membership, factories, or behavior.
Both views are now derived from one validated descriptor set.
- **Make identity JSON projection an implicit default.** Rejected
because omission would silently assert that the database representation
is already lossless. Every descriptor states the transitional behavior
explicitly.
- **Activate JSON projection hooks in this PR.** Rejected so
registry/authoring migration and representation changes remain
independently reviewable. TML-3063 owns the observable hard cut.
## Checklist
- [x] All commits are signed off (`git commit -s`) per the
[DCO](CONTRIBUTING.md#developer-certificate-of-origin-dco).
- [x] I read [CONTRIBUTING.md](CONTRIBUTING.md) and the change is scoped
to one logical concern.
- [x] Tests are updated for the public protocols, composition boundary,
extensions, declaration portability, and unchanged rendering behavior.
- [x] The PR title is in `TML-NNNN: <sentence-case title>` form and
names the concrete deliverable.
- [x] The **Skill update** section is filled in.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added PostgreSQL- and SQLite-specific codec descriptor authoring APIs,
including descriptor/codec registries and validation.
* Adapters now use provided target-specific codec descriptor sets to
keep runtime and control-plane behavior consistent.
* Improved native type handling and JSON projection behavior for
parameterized and scalar-array scenarios.
* **Bug Fixes**
* PostGIS geometry codecs now support optional SRID, with updated
typing, validation, and output rendering.
* **Documentation**
* Expanded codec authoring guide, target READMEs, and 0.16 → 0.17
upgrade instructions for adopting the new descriptor model.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Signed-off-by: Alexey Orlenko's AI Agent <robot@aqrln.net>1 parent 4e51fef commit e0e739c
98 files changed
Lines changed: 5201 additions & 662 deletions
File tree
- docs/reference
- packages
- 2-sql/2-authoring/contract-ts/src
- 3-extensions
- arktype-json
- src/core
- test
- pgvector
- src/core
- test
- postgis
- src/core
- test
- 3-targets
- 3-targets
- postgres
- src
- core
- exports
- test
- sqlite
- src
- core
- exports
- test
- 6-adapters
- postgres
- src
- core
- exports
- test
- migrations
- sqlite
- src
- core
- exports
- test
- projects/codec-json-projections
- slices/02-target-codec-descriptor-foundations
- dispatches
- skills/extension-author/prisma-next-extension-upgrade/upgrades/0.16-to-0.17
Some content is hidden
Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.
Large diffs are not rendered by default.
Lines changed: 9 additions & 2 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
21 | 21 | | |
22 | 22 | | |
23 | 23 | | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
24 | 31 | | |
25 | | - | |
| 32 | + | |
26 | 33 | | |
27 | | - | |
| 34 | + | |
28 | 35 | | |
29 | 36 | | |
30 | 37 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
21 | 21 | | |
22 | 22 | | |
23 | 23 | | |
| 24 | + | |
24 | 25 | | |
25 | 26 | | |
26 | 27 | | |
| |||
Lines changed: 13 additions & 4 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
13 | 13 | | |
14 | 14 | | |
15 | 15 | | |
16 | | - | |
17 | 16 | | |
18 | | - | |
19 | 17 | | |
20 | 18 | | |
21 | 19 | | |
22 | 20 | | |
23 | 21 | | |
24 | 22 | | |
25 | 23 | | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
26 | 29 | | |
27 | 30 | | |
28 | 31 | | |
| |||
208 | 211 | | |
209 | 212 | | |
210 | 213 | | |
211 | | - | |
| 214 | + | |
| 215 | + | |
| 216 | + | |
| 217 | + | |
| 218 | + | |
| 219 | + | |
| 220 | + | |
212 | 221 | | |
213 | 222 | | |
214 | 223 | | |
| |||
284 | 293 | | |
285 | 294 | | |
286 | 295 | | |
287 | | - | |
| 296 | + | |
Lines changed: 28 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
Lines changed: 52 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
26 | 26 | | |
27 | 27 | | |
28 | 28 | | |
| 29 | + | |
29 | 30 | | |
30 | 31 | | |
31 | 32 | | |
| |||
36 | 37 | | |
37 | 38 | | |
38 | 39 | | |
39 | | - | |
40 | 40 | | |
41 | 41 | | |
42 | 42 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
4 | 4 | | |
5 | 5 | | |
6 | 6 | | |
7 | | - | |
| 7 | + | |
8 | 8 | | |
9 | 9 | | |
10 | 10 | | |
| |||
14 | 14 | | |
15 | 15 | | |
16 | 16 | | |
17 | | - | |
18 | 17 | | |
19 | 18 | | |
20 | 19 | | |
21 | 20 | | |
22 | 21 | | |
23 | 22 | | |
24 | | - | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
25 | 28 | | |
26 | 29 | | |
27 | 30 | | |
| |||
147 | 150 | | |
148 | 151 | | |
149 | 152 | | |
150 | | - | |
| 153 | + | |
| 154 | + | |
| 155 | + | |
| 156 | + | |
| 157 | + | |
| 158 | + | |
| 159 | + | |
151 | 160 | | |
152 | 161 | | |
153 | 162 | | |
| |||
180 | 189 | | |
181 | 190 | | |
182 | 191 | | |
183 | | - | |
| 192 | + | |
Lines changed: 8 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
Lines changed: 29 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
0 commit comments