-
Notifications
You must be signed in to change notification settings - Fork 0
Expand file tree
/
Copy pathTaskfile.yml
More file actions
242 lines (210 loc) · 11.2 KB
/
Copy pathTaskfile.yml
File metadata and controls
242 lines (210 loc) · 11.2 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
version: '3'
vars:
ENV: .env.local
tasks:
run:
desc: Start the development server (default ENV=.env.local)
summary: |
Starts the Jetty server on port 5001.
Override the env file: task run ENV=.env.aws
cmd: ./bin/run --environment={{.ENV}}
test:
desc: Run the full test suite
cmd: ./bin/test
test:summary:
desc: Run tests and print only failures (names + assertions, no stack traces)
cmd: ./bin/test 2>&1 | grep -E "^(FAIL|ERROR) in|expected:| actual:"
db:migrate:
desc: Run pending database migrations (default ENV=.env.local)
cmd: ./bin/db-migrate --environment={{.ENV}}
db:reset:
desc: Drop all tables and re-run migrations — DESTRUCTIVE (default ENV=.env.local)
prompt: This will drop all tables on the target database. Continue?
cmd: ./bin/db-reset --environment={{.ENV}}
db:connect:
desc: Open a psql shell to the target database (default ENV=.env.local)
cmd: ./bin/psql-connect --environment={{.ENV}}
db:squash:list:
desc: 'Preview the migrations a squash would collapse — read-only (usage: task db:squash:list -- FROM TO)'
cmd: ./bin/db-squash squash-list --environment={{.ENV}} -- {{.CLI_ARGS}}
db:squash:create:
desc: 'Collapse migrations [FROM TO] into one file — REWRITES REPO FILES (usage: task db:squash:create -- FROM TO NAME)'
prompt: This deletes the original migration files and writes one squashed file. Continue?
cmd: ./bin/db-squash squash-create --environment={{.ENV}} -- {{.CLI_ARGS}}
db:squash:apply:
desc: 'Reconcile ONE database''s schema_migrations after a squash — run per environment (usage: task db:squash:apply ENV=.env.aws -- FROM TO NAME)'
prompt: This rewrites schema_migrations on the target database. Continue?
cmd: ./bin/db-squash squash-apply --environment={{.ENV}} -- {{.CLI_ARGS}}
build:uberjar:
desc: Compile and package the standalone JAR
cmd: ./bin/uberjar
build:docker:
desc: Build the Docker image (requires build:uberjar to have run first)
cmd: ./bin/docker-build
deploy:
desc: Run tests, migrate the prod DB, build the uberjar, and deploy to Fly.io
summary: |
Migrations run against prod (.env.fly.prod) BEFORE the Fly deploy so the
schema is ready before the new code boots — a deploy that ships code ahead
of its migrations is what broke /recipes/scrape in prod. Override the
migration target with DEPLOY_ENV if deploying elsewhere.
vars:
DEPLOY_ENV: '{{.DEPLOY_ENV | default ".env.fly.prod"}}'
cmds:
- task: test
- task: build:uberjar
- task: db:migrate
vars:
ENV: '{{.DEPLOY_ENV}}'
- ./bin/deploy-image
env:up:
desc: Provision an ephemeral Fly.io test environment (backend + frontend; NAME defaults to the git branch)
summary: |
Creates a Neon branch (eph-<slug>), deploys a Fly app (kal-eph-<slug>) against
real dependencies, smoke-tests it, and runs the full Checkly spec suite against
the live env. Requires .env.fly.staging; the Checkly step is skipped with a
warning unless CHECKLY_API_KEY/CHECKLY_ACCOUNT_ID and AUTH0_CLIENT_ID/SECRET
are set. Name it explicitly: task env:up NAME=pr-42
env:
TENANT: '{{.TENANT}}'
cmd: ./scripts/ephemeral/up {{if .NAME}}--name={{.NAME}}{{end}}
env:down:
desc: Tear down an ephemeral Fly.io test environment (NAME defaults to the git branch)
cmd: ./scripts/ephemeral/down {{if .NAME}}--name={{.NAME}}{{end}}
env:test:
desc: Unit-test the ephemeral slug helpers (no services required)
cmd: ./scripts/ephemeral/lib_test.sh
env:checkly:
desc: Run the full Checkly spec suite against an ephemeral env (NAME defaults to the git branch)
summary: |
Points the Monitoring-as-Code checks at https://kal-eph-<slug>.fly.dev and runs
a one-off `checkly test` in Checkly's cloud. Needs CHECKLY_API_KEY/CHECKLY_ACCOUNT_ID
and AUTH0_CLIENT_ID/SECRET (skipped with a warning otherwise).
cmd: ./scripts/ephemeral/checkly-test {{if .NAME}}--name={{.NAME}}{{end}}
# ephemeral:* mirror the env:* tasks but demand an explicit NAME (no git-branch
# fallback), so you always target a named environment on purpose.
ephemeral:up:
desc: Provision a named ephemeral Fly.io test environment end-to-end (requires NAME)
summary: |
Like env:up but pinned to an explicit NAME instead of the git branch.
Creates a Neon branch (eph-<NAME>), deploys a Fly app (kal-eph-<NAME>) against
real dependencies, smoke-tests it, and runs the full Checkly spec suite. Requires
.env.fly.staging; the Checkly step is skipped with a warning unless
CHECKLY_API_KEY/CHECKLY_ACCOUNT_ID and AUTH0_CLIENT_ID/SECRET are set.
TENANT picks which tenant hostname (from resources/tenants.json) the env
impersonates; defaults to DEFAULT_TENANT (andrewslai.com).
Usage: task ephemeral:up NAME=pr-42 TENANT=caheriaguilar.com
requires:
vars: [NAME]
env:
TENANT: '{{.TENANT}}'
cmd: ./scripts/ephemeral/up --name={{.NAME}}
ephemeral:down:
desc: Tear down an ephemeral Fly.io test environment (NAME optional; prompts if omitted)
summary: |
With no NAME, auto-detects the deployed kal-eph-* Fly apps and prompts you to
pick one to tear down (auto-selects when there's exactly one). Pass NAME to skip
the prompt. Usage: task ephemeral:down OR task ephemeral:down NAME=pr-42
interactive: true
cmd: ./scripts/ephemeral/down {{if .NAME}}--name={{.NAME}}{{end}}
ephemeral:provision-db:
desc: Create and migrate the Neon branch for a named ephemeral env (requires NAME)
requires:
vars: [NAME]
cmd: ./scripts/ephemeral/provision-db --name={{.NAME}}
ephemeral:build-frontend:
desc: Build the frontend and sync it to an ephemeral env's S3 prefix (NAME optional; prompts if omitted)
summary: |
With no NAME, auto-detects the deployed kal-eph-* Fly apps and prompts you to
pick which env's S3 prefix to sync to (auto-selects when there's exactly one).
Pass NAME to skip the prompt.
Usage: task ephemeral:build-frontend OR task ephemeral:build-frontend NAME=pr-42
interactive: true
cmd: ./scripts/ephemeral/build-frontend {{if .NAME}}--name={{.NAME}}{{end}}
ephemeral:deploy-app:
desc: Build and deploy the backend to a named ephemeral Fly app (requires NAME)
requires:
vars: [NAME]
env:
TENANT: '{{.TENANT}}'
cmd: ./scripts/ephemeral/deploy-app --name={{.NAME}}
ephemeral:smoke-test:
desc: Smoke-test a deployed ephemeral env (NAME optional; prompts if omitted)
summary: |
With no NAME, auto-detects the deployed kal-eph-* Fly apps and prompts you to
pick one to smoke-test (auto-selects when there's exactly one). Pass NAME to skip
the prompt. Usage: task ephemeral:smoke-test OR task ephemeral:smoke-test NAME=pr-42
interactive: true
cmd: ./scripts/ephemeral/smoke-test {{if .NAME}}--name={{.NAME}}{{end}}
ephemeral:checkly-test:
desc: Run the full Checkly spec suite against an ephemeral env (NAME optional; prompts if omitted)
summary: |
Points the Monitoring-as-Code checks at https://kal-eph-<NAME>.fly.dev and runs
a one-off `checkly test` in Checkly's cloud. Needs CHECKLY_API_KEY/CHECKLY_ACCOUNT_ID
and AUTH0_CLIENT_ID/SECRET (skipped with a warning otherwise).
Unlike the other ephemeral:* tasks, NAME is optional here: with no NAME the
task auto-detects the deployed kal-eph-* Fly apps and prompts you to pick one
(auto-selects when there's exactly one). Pass NAME to skip the prompt.
Usage: task ephemeral:checkly-test OR task ephemeral:checkly-test NAME=pr-42
interactive: true
cmd: ./scripts/ephemeral/checkly-test {{if .NAME}}--name={{.NAME}}{{end}}
ephemeral:reap:
desc: Reap orphaned per-env media buckets left by failed teardowns (dry-run unless APPLY=1)
summary: |
Finds every kal-eph-*-media bucket whose Fly app (kal-eph-<slug>) no longer
exists — the residue of a crashed `down` or killed CI job — and deletes it.
Each orphan holds a live read credential on prod media and counts against
the ~100/account bucket ceiling. Dry-run by default (prints the kill list);
pass APPLY=1 to actually empty+delete. A grace period (REAP_GRACE_HOURS,
default 6) protects a bucket whose env is still mid-provision. Run by hand
when orphans are suspected; auto-scheduling is a deferred follow-up.
Usage: task ephemeral:reap OR task ephemeral:reap APPLY=1
cmd: ./scripts/ephemeral/reap-orphan-buckets {{if eq .APPLY "1"}}--apply{{else}}--dry-run{{end}}
media:consolidate:
desc: Consolidate every per-tenant media bucket into kal-media-prod (Phase-2 prerequisite)
summary: |
Server-side `aws s3 sync` of each per-tenant bucket's media/ prefix (from
resources/tenants.json) into s3://kal-media-prod/media/. UUID-keyed objects
merge losslessly (no --delete). Idempotent — run incrementally before the
maintenance window, then once more in-window to catch the delta. Verifies a
sample of consolidated objects and fails loudly on any miss.
Usage: task media:consolidate OR PROD_MEDIA_BUCKET=kal-media-prod task media:consolidate
cmd: ./scripts/media/consolidate-buckets
media:verify-resize:
desc: Upload a photo and assert a rendition appears — end-to-end resize fitness function
summary: |
Drives the whole resize chain against a live env: upload -> write-location
s3://bucket/key notify -> SNS -> SQS -> resizer Lambda -> rendition served
back. Polls GET /v2/photos/<id>/gallery.<ext> until 200 or timeout and fails
loudly naming the broken hop. Use as the final post-flip gate in the Phase-2
runbook (replacing the eyeball curl) and on a schedule against prod.
Needs AUTH0_CLIENT_ID/SECRET. TARGET_URL selects the env (default prod).
Usage: TARGET_URL=https://andrewslai.com task media:verify-resize
cmd: ./scripts/media/verify-resize-roundtrip {{if .URL}}--url={{.URL}}{{end}}
media:reconcile:
desc: Offline reconciliation/reclamation of the media store (dry-run unless APPLY=1)
summary: |
Diffs the source of truth (live photo_versions rows) against the stored
objects: orphans = stored - referenced (quarantined to trash/, never hard
deleted), dangling = referenced - stored (data-loss alert), mismatched =
rows whose content_hash disagrees with the stored bytes (integrity alert).
Gated: refuses on a suspicious index shrink or a failed index-health check.
The set-math + gates are typed, unit-tested Clojure; this only launches it.
Required env: KALEIDOSCOPE_MEDIA_BUCKET, RECONCILE_STORED_KEYS (keys file
from the S3 Inventory export), KALEIDOSCOPE_DB_*. NEVER run against a
corrupted index — restore from PITR first. Monthly cadence.
Usage: task media:reconcile OR task media:reconcile APPLY=1
env:
RECONCILE_APPLY: '{{if eq .APPLY "1"}}1{{end}}'
cmd: ./scripts/media/reconcile
deploy:otel-collector:
desc: Deploy the OTEL collector to Fly.io
dir: iac/otel-collector
cmd: fly deploy
deploy:checkly:
desc: Deploy Checkly checks (monitoring as code)
dir: checkly
cmd: npx checkly deploy
clean:
desc: Remove build artifacts
cmd: ./bin/clean