Skip to content

Commit 82e6d06

Browse files
avalleteclaude
andauthored
fix(cli): provision platform baseline in declarative baseline catalog (#5521)
Refs CLI-1601 — https://linear.app/supabase/issue/CLI-1601/support-auth-dependencies-in-shadow-db-migrations Follow-up to #5515. That PR taught the declarative apply path to provision the Supabase platform baseline (auth/storage/realtime) on the shadow before applying declarative schemas. This fixes a gap it left in the `generate` → `sync`-with-no-migrations handoff. ## The bug `getGenerateBaselineCatalogRef` exports `.temp/pgdelta/catalog-baseline-<version>.json` immediately after creating the shadow — **before** any platform setup runs — so the cached baseline represents a bare postgres image. That same file is reused by `getMigrationsCatalogRef` as the diff **source** when there are zero local migrations. After #5515, the declarative **target** is built on top of the platform baseline. So a no-migration `sync` diffs: - source: bare postgres image - target: platform baseline + declarative schema instead of the intended: - source: platform baseline - target: platform baseline + declarative schema Platform-managed objects (`auth`, `storage`, `realtime`, grants, functions, …) no longer cancel and surface as spurious additions in the generated migration — e.g. a single declarative `public.profiles` table referencing `auth.users` produces a migration that also tries to create the `auth`/`storage`/`realtime` platform objects. ## The fix Provision the platform baseline in `getGenerateBaselineCatalogRef` **before** exporting, so the baseline catalog consistently means "platform baseline, no user migrations" — identical to `diff.MigrateShadowDatabase` with zero migrations, and in parity with the declarative target. The now-redundant second `setupShadowDatabase` call in `Generate`'s cache-warm path is removed since the reused shadow already has the baseline (it was previously provisioning the platform twice). This also fixes a latent `generate` issue: with a bare baseline, `generate` would have emitted platform schemas into the user's declarative files. ## Tests - `TestGetGenerateBaselineCatalogRefSetsUpPlatformBaseline` — pins the setup-before-export ordering. - `TestGenerateThenSyncWithNoMigrationsCancelsPlatformObjects` — full generate → no-migration sync flow through the public command functions, asserting only the user's table is generated and platform objects cancel. Docker/pg-delta seams are stubbed (the established cli-go pattern), so it runs in the standard `go test ./...` CI job. `diff.DiffPgDeltaRef` is now an injectable package var to allow diffing through the public path without the real pg-delta runtime. https://claude.ai/code/session_01ACrX8NXcsYLENnCRXAub3S --- _Generated by [Claude Code](https://claude.ai/code/session_01ACrX8NXcsYLENnCRXAub3S)_ --------- Co-authored-by: Claude <noreply@anthropic.com>
1 parent f54aba0 commit 82e6d06

3 files changed

Lines changed: 511 additions & 33 deletions

File tree

apps/cli-go/internal/db/declarative/declarative.go

Lines changed: 158 additions & 25 deletions
Original file line numberDiff line numberDiff line change
@@ -31,10 +31,16 @@ const (
3131
// pgDeltaTempDir namespaces pg-delta artifacts under .temp to make ownership
3232
// and cleanup intent explicit.
3333
pgDeltaTempDir = "pgdelta"
34-
// baselineCatalogName caches the catalog of an empty shadow database.
34+
// baselineCatalogName caches the catalog of a shadow database with the Supabase
35+
// platform baseline (auth/storage/realtime) provisioned but no user migrations
36+
// applied — equivalent to diff.MigrateShadowDatabase with zero migrations.
3537
//
36-
// It is used as the "source" baseline when generating declarative files from
37-
// a real database target.
38+
// It is used as the "source" baseline both when generating declarative files
39+
// from a real database target and when syncing with no local migrations, so it
40+
// must stay in parity with the declarative target's platform baseline. The "%s"
41+
// is a key (see baselineCatalogKey) derived from the image plus every setup
42+
// input that shapes the baseline, so config/roles changes self-invalidate the
43+
// cache rather than reusing a stale snapshot.
3844
baselineCatalogName = "catalog-baseline-%s.json"
3945
// declarativeCatalogName stores catalogs keyed by declarative-content hash.
4046
declarativeCatalogName = "catalog-%s-declarative-%s-%d.json"
@@ -58,11 +64,19 @@ var (
5864
exportCatalog = diff.ExportCatalogPgDelta
5965
applyDeclarative = pgdelta.ApplyDeclarative
6066
declarativeExportRef = diff.DeclarativeExportPgDeltaRef
67+
// diffPgDeltaRef diffs a source catalog against a target catalog. It is a
68+
// package var so tests can exercise the full generate -> sync flow without the
69+
// real pg-delta runtime.
70+
diffPgDeltaRef = diff.DiffPgDeltaRef
6171
// setupShadowDatabase provisions the Supabase platform baseline (auth/storage/
6272
// realtime) on a shadow database before declarative schemas are applied, so
6373
// Supabase-managed dependencies (auth.sessions, auth.jwt(), ...) resolve. It is
6474
// a package var so tests can inject a no-op without a real shadow database.
6575
setupShadowDatabase = diff.SetupShadowDatabase
76+
// createShadow provisions a healthy shadow database container. It is a package
77+
// var so tests can exercise the baseline/migrations/declarative paths without a
78+
// real Docker daemon.
79+
createShadow = createShadowContainer
6680
// generateBaselineCatalogRefResolver allows Generate to reuse a freshly
6781
// provisioned baseline shadow for declarative cache warmup.
6882
generateBaselineCatalogRefResolver = getGenerateBaselineCatalogRef
@@ -123,14 +137,10 @@ func Generate(ctx context.Context, schema []string, config pgconn.Config, overwr
123137
// can reuse it without provisioning another shadow database.
124138
if !noCache {
125139
if baseline.shadow != nil {
126-
// The baseline catalog was already exported from the empty image
127-
// baseline above. Set up the platform baseline on the reused shadow
128-
// before applying declarative schemas so Supabase-managed dependencies
129-
// (auth.sessions, auth.jwt(), ...) resolve during cache warmup.
130-
if err := setupShadowDatabase(ctx, baseline.shadow.container, fsys, options...); err != nil {
131-
return err
132-
}
133-
hash, err := hashDeclarativeSchemas(fsys)
140+
// The reused baseline shadow already has the platform baseline
141+
// provisioned (getGenerateBaselineCatalogRef), so apply declarative
142+
// schemas directly on top of it without setting it up again.
143+
hash, err := declarativeCatalogCacheKey(fsys)
134144
if err != nil {
135145
return err
136146
}
@@ -172,7 +182,7 @@ func DiffDeclarativeToMigrations(ctx context.Context, schema []string, noCache b
172182
if err != nil {
173183
return nil, err
174184
}
175-
out, err := diff.DiffPgDeltaRef(ctx, sourceRef, targetRef, schema, pgDeltaFormatOptions(), options...)
185+
out, err := diffPgDeltaRef(ctx, sourceRef, targetRef, schema, pgDeltaFormatOptions(), options...)
176186
if err != nil {
177187
return nil, err
178188
}
@@ -294,7 +304,10 @@ func updateDeclarativeSchemaPathsConfig(fsys afero.Fs) error {
294304
}
295305

296306
func getGenerateBaselineCatalogRef(ctx context.Context, noCache bool, fsys afero.Fs, options ...func(*pgx.ConnConfig)) (generateBaselineCatalogRef, error) {
297-
cachePath := filepath.Join(pgDeltaTempPath(), fmt.Sprintf(baselineCatalogName, baselineVersionToken()))
307+
cachePath, err := baselineCatalogPath(fsys)
308+
if err != nil {
309+
return generateBaselineCatalogRef{}, err
310+
}
298311
if !noCache {
299312
if ok, err := afero.Exists(fsys, cachePath); err == nil && ok {
300313
return generateBaselineCatalogRef{ref: cachePath}, nil
@@ -308,6 +321,18 @@ func getGenerateBaselineCatalogRef(ctx context.Context, noCache bool, fsys afero
308321
container: shadowID,
309322
config: config,
310323
}
324+
// Provision the Supabase platform baseline before exporting so the baseline
325+
// catalog represents "platform baseline, no user migrations" — the same
326+
// semantics as diff.MigrateShadowDatabase with zero migrations. This baseline is
327+
// reused as the diff source by both Generate (against the live database) and
328+
// sync-with-no-migrations (getMigrationsCatalogRef). Its starting point must
329+
// match the declarative target, which also sets up the platform baseline;
330+
// otherwise platform objects (auth/storage/realtime) surface as spurious
331+
// additions in generated migrations.
332+
if err := setupShadowDatabase(ctx, shadow.container, fsys, options...); err != nil {
333+
shadow.cleanup()
334+
return generateBaselineCatalogRef{}, err
335+
}
311336
snapshot, err := exportCatalog(ctx, utils.ToPostgresURL(config), "postgres", options...)
312337
if err != nil {
313338
shadow.cleanup()
@@ -345,21 +370,31 @@ func getMigrationsCatalogRef(ctx context.Context, noCache bool, fsys afero.Fs, p
345370
if err != nil {
346371
return "", err
347372
}
348-
// For sync with no local migrations, reuse an existing baseline
349-
// snapshot instead of provisioning a fresh shadow database.
350-
if !noCache && len(migrations) == 0 {
351-
baselinePath := filepath.Join(pgDeltaTempPath(), fmt.Sprintf(baselineCatalogName, baselineVersionToken()))
352-
if ok, err := afero.Exists(fsys, baselinePath); err != nil {
373+
// With no local migrations, the migrations catalog is exactly the platform
374+
// baseline, so it is cached under the setup-keyed baseline path rather than the
375+
// migrations-hash cache. The migrations-hash cache is not setup-aware, so an
376+
// older empty-migrations snapshot from a different platform setup must not be
377+
// reused as the no-migration sync source.
378+
zeroMigrations := len(migrations) == 0
379+
var baselinePath string
380+
if zeroMigrations {
381+
baselinePath, err = baselineCatalogPath(fsys)
382+
if err != nil {
353383
return "", err
354-
} else if ok {
355-
return baselinePath, nil
384+
}
385+
if !noCache {
386+
if ok, err := afero.Exists(fsys, baselinePath); err != nil {
387+
return "", err
388+
} else if ok {
389+
return baselinePath, nil
390+
}
356391
}
357392
}
358393
hash, err := pgcache.HashMigrations(fsys)
359394
if err != nil {
360395
return "", err
361396
}
362-
if !noCache {
397+
if !noCache && !zeroMigrations {
363398
if cachePath, ok, err := pgcache.ResolveMigrationCatalogPath(fsys, hash, prefix); err != nil {
364399
return "", err
365400
} else if ok {
@@ -381,13 +416,23 @@ func getMigrationsCatalogRef(ctx context.Context, noCache bool, fsys afero.Fs, p
381416
if noCache {
382417
return writeTempCatalog(fsys, noCacheMigrationsCatalogPath, snapshot)
383418
}
419+
if zeroMigrations {
420+
// MigrateShadowDatabase with zero migrations == the platform baseline.
421+
if err := ensureTempDir(fsys); err != nil {
422+
return "", err
423+
}
424+
if err := utils.WriteFile(baselinePath, []byte(snapshot), fsys); err != nil {
425+
return "", err
426+
}
427+
return baselinePath, nil
428+
}
384429
return pgcache.WriteMigrationCatalogSnapshot(fsys, prefix, hash, snapshot)
385430
}
386431

387432
// getDeclarativeCatalogRef applies local declarative files to a shadow database
388433
// and exports the resulting catalog for diffing.
389434
func getDeclarativeCatalogRef(ctx context.Context, noCache bool, fsys afero.Fs, options ...func(*pgx.ConnConfig)) (string, error) {
390-
hash, err := hashDeclarativeSchemas(fsys)
435+
hash, err := declarativeCatalogCacheKey(fsys)
391436
if err != nil {
392437
return "", err
393438
}
@@ -439,9 +484,9 @@ func writeDeclarativeCatalogFromConfig(ctx context.Context, config pgconn.Config
439484
return path, nil
440485
}
441486

442-
// createShadow provisions and health-checks the temporary Postgres container
443-
// used by declarative conversion and diff operations.
444-
func createShadow(ctx context.Context) (string, pgconn.Config, error) {
487+
// createShadowContainer provisions and health-checks the temporary Postgres
488+
// container used by declarative conversion and diff operations.
489+
func createShadowContainer(ctx context.Context) (string, pgconn.Config, error) {
445490
fmt.Fprintln(os.Stderr, "Creating shadow database...")
446491
shadow, err := diff.CreateShadowDatabase(ctx, utils.Config.Db.ShadowPort)
447492
if err != nil {
@@ -628,6 +673,94 @@ func baselineVersionToken() string {
628673
return catalogPrefixRegexp.ReplaceAllString(image, "-")
629674
}
630675

676+
// setupInputsToken hashes every project input that start.SetupDatabase consumes
677+
// and that therefore shapes the platform baseline:
678+
//
679+
// - the Postgres image (initSchema content);
680+
// - the service toggles that gate initSchema — auth/storage/realtime;
681+
// - api.auto_expose_new_tables (ApplyApiPrivileges default ACLs);
682+
// - vault secret names (UpsertVaultSecrets);
683+
// - supabase/roles.sql (SeedGlobals).
684+
//
685+
// Every catalog produced in this flow is "platform baseline + {nothing | migrations
686+
// | declarative}", so each cache folds this token into its key and self-invalidates
687+
// when setup changes instead of reusing a snapshot from a different baseline.
688+
func setupInputsToken(fsys afero.Fs) (string, error) {
689+
h := sha256.New()
690+
fmt.Fprintln(h, baselineVersionToken())
691+
// initSchema conditionally provisions these service schemas.
692+
fmt.Fprintf(h, "auth=%t storage=%t realtime=%t\n",
693+
utils.Config.Auth.Enabled, utils.Config.Storage.Enabled, utils.Config.Realtime.Enabled)
694+
// api.auto_expose_new_tables drives ApplyApiPrivileges (default ACLs).
695+
if v := utils.Config.Api.AutoExposeNewTables; v != nil {
696+
fmt.Fprintf(h, "auto_expose_new_tables=%t\n", *v)
697+
} else {
698+
fmt.Fprintln(h, "auto_expose_new_tables=unset")
699+
}
700+
// Vault secrets are created during setup; key on their names.
701+
names := make([]string, 0, len(utils.Config.Db.Vault))
702+
for name := range utils.Config.Db.Vault {
703+
names = append(names, name)
704+
}
705+
sort.Strings(names)
706+
for _, name := range names {
707+
fmt.Fprintf(h, "vault=%s\n", name)
708+
}
709+
// supabase/roles.sql is seeded into the baseline.
710+
roles, err := afero.ReadFile(fsys, utils.CustomRolesPath)
711+
if err != nil && !errors.Is(err, os.ErrNotExist) {
712+
return "", err
713+
}
714+
if _, err := h.Write(roles); err != nil {
715+
return "", err
716+
}
717+
return hex.EncodeToString(h.Sum(nil))[:12], nil
718+
}
719+
720+
// baselineCatalogKey derives the cache key for the platform baseline catalog.
721+
//
722+
// Keying only by image would let a stale baseline — produced by a pre-platform-
723+
// baseline CLI, a different image, or different service/api/vault/roles config — be
724+
// reused as the no-migration diff source, leaking spurious objects into generated
725+
// migrations until .temp/pgdelta is cleared. The image token stays as a human-
726+
// readable prefix; old bare-baseline files keyed by the token alone no longer
727+
// match, so they are never reused.
728+
func baselineCatalogKey(fsys afero.Fs) (string, error) {
729+
token, err := setupInputsToken(fsys)
730+
if err != nil {
731+
return "", err
732+
}
733+
return baselineVersionToken() + "-" + token, nil
734+
}
735+
736+
// baselineCatalogPath returns the on-disk path of the platform baseline catalog
737+
// for the current project inputs. Both the generate writer and the no-migration
738+
// sync reader resolve the path through this helper so they always agree.
739+
func baselineCatalogPath(fsys afero.Fs) (string, error) {
740+
key, err := baselineCatalogKey(fsys)
741+
if err != nil {
742+
return "", err
743+
}
744+
return filepath.Join(pgDeltaTempPath(), fmt.Sprintf(baselineCatalogName, key)), nil
745+
}
746+
747+
// declarativeCatalogCacheKey keys the warmed declarative target catalog by both the
748+
// declarative SQL files and the setup inputs. The target is the platform baseline
749+
// plus the declarative schema, so a change to either must invalidate it; otherwise
750+
// sync could pair a freshly keyed source baseline with a target warmed under a
751+
// different setup, emitting platform/config-only differences as user migrations.
752+
func declarativeCatalogCacheKey(fsys afero.Fs) (string, error) {
753+
schemaHash, err := hashDeclarativeSchemas(fsys)
754+
if err != nil {
755+
return "", err
756+
}
757+
setup, err := setupInputsToken(fsys)
758+
if err != nil {
759+
return "", err
760+
}
761+
return setup + "-" + schemaHash, nil
762+
}
763+
631764
func sanitizedCatalogPrefix(prefix string) string {
632765
prefix = strings.TrimSpace(prefix)
633766
if len(prefix) == 0 {

0 commit comments

Comments
 (0)