You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
// Schema is auto-created; capture triggers are wired automatically.
218
213
db.exec(`CREATE TABLE IF NOT EXISTS orders (
@@ -222,13 +217,13 @@ db.exec(`CREATE TABLE IF NOT EXISTS orders (
222
217
ts INTEGER NOT NULL DEFAULT (unixepoch())
223
218
)`);
224
219
225
-
// Every INSERT is captured and shipped to the control plane in < 2 s.
220
+
// Every INSERT is captured and shipped in < 2 s.
226
221
exportfunctionplaceOrder(item, qty) {
227
222
db.run('INSERT INTO orders (item, qty) VALUES (?, ?)', [item, qty]);
228
223
returndb.lastInsertRowid;
229
224
}
230
225
231
-
// Health endpoint — wire to Cloud Run liveness probe.
226
+
// Health endpoint.
232
227
exportfunctionhealth() {
233
228
return {
234
229
healthy:db.backupHealthy,
@@ -241,66 +236,30 @@ export function health() {
241
236
process.on('SIGTERM', () =>db.close());
242
237
```
243
238
244
-
**What happens on GCP:**
245
-
- Deploy to **Cloud Run** (scales to zero — Arkilian's 2-second poll loop costs nothing at idle).
246
-
- Point `ARKILIAN_SIGNED_URL_ENDPOINT` at a **Cloud Function** that issues GCS signed URLs.
247
-
- Every hourly snapshot lands in a **Cloud Storage** bucket (`gs://arkilian-backups/<tenant>/`).
248
-
- The WAL stream feeds your control plane, which fans out to **BigQuery** for analytics.
249
-
250
239
---
251
240
252
-
### 2 — Real-time analytics pipeline into BigQuery
241
+
### 2 — Real-time CDC Pipeline
253
242
254
-
Arkilian's WAL push endpoint ships every row change as replayable SQL within
255
-
2 seconds of commit. Wire your control plane to publish those payloads onto
256
-
**Pub/Sub** and let a Dataflow pipeline hydrate **BigQuery** in near-real time.
243
+
Configure the background worker with your `ARKILIAN_DATABASE_TOKEN` and endpoints obtained from [arkilian.com](https://arkilian.com) to stream raw row operations in real time.
-- BigQuery scheduled query: materialize the orders table from CDC
284
-
SELECT
285
-
JSON_VALUE(params, '$[0]') AS item,
286
-
CAST(JSON_VALUE(params, '$[1]') AS INT64) AS qty,
287
-
received_at
288
-
FROM`project.arkilian_cdc.raw_events`
289
-
WHERE sql LIKE'INSERT INTO orders%'
290
-
ORDER BY received_at;
291
-
```
248
+
// Get your configuration and API token from https://arkilian.com
249
+
consttoken=process.env.ARKILIAN_DATABASE_TOKEN;
250
+
constdb=newArkilian(token, 'app.sqlite');
292
251
293
-
The result: **sub-5-second latency** from SQLite write to BigQuery row — without
294
-
Kafka, Debezium, or a managed database. The entire pipeline is SQLite on the
295
-
edge, a Cloud Function in the middle, and BigQuery at the end.
252
+
db.exec(`CREATE TABLE IF NOT EXISTS users (
253
+
id INTEGER PRIMARY KEY,
254
+
email TEXT NOT NULL UNIQUE
255
+
)`);
256
+
```
296
257
297
258
---
298
259
299
-
### 3 — Offline-first mobile backend (Go / Cloud Run)
260
+
### 3 — Offline-first Go Backend
300
261
301
-
The Go binding lets you embed Arkilian directly into a Go service with zero CGO
302
-
overhead beyond the initial open. Here a game server persists per-player state
303
-
locally and replicates automatically.
262
+
Link the native library directly into your Go binaries. Acquire the client SDK assets and environment configuration templates from [arkilian.com](https://arkilian.com).
304
263
305
264
```go
306
265
// main.go
@@ -320,9 +279,10 @@ import (
320
279
321
280
funcmain() {
322
281
vardb *C.arkilian
323
-
path:= C.CString("players.sqlite")
282
+
path:= C.CString("app.sqlite")
324
283
defer C.free(unsafe.Pointer(path))
325
284
285
+
// Initialize using configuration obtained from https://arkilian.com
326
286
if C.db_init(&db, path) != 0 {
327
287
log.Fatal("db_init failed")
328
288
}
@@ -335,59 +295,30 @@ func main() {
335
295
)`)
336
296
defer C.free(unsafe.Pointer(sql))
337
297
C.db_exec(db, sql)
338
-
339
-
// Every score update ships to GCS within 2 seconds.
340
-
upd:= C.CString("UPDATE players SET score = score + 1 WHERE id = 1")
341
-
defer C.free(unsafe.Pointer(upd))
342
-
C.db_exec(db, upd)
343
-
344
-
// Monitoring
345
-
fmt.Printf("queue=%d healthy=%d\n",
346
-
C.db_backup_queue_depth(db),
347
-
C.db_backup_is_healthy(db))
348
298
}
349
299
```
350
300
351
-
**Deploy pattern on GCP:**
352
-
- Build to a Docker image → push to **Artifact Registry** → run on **Cloud Run** or **GKE**.
353
-
- Each player shard is a Cloud Run instance with its own `players.sqlite`.
354
-
- Arkilian replicates to **Cloud Storage**; a Cloud Scheduler job triggers hydration on instance spin-up.
355
-
- 5,000 shards = 5,000 independent SQLite files, each replicating at 2-second cadence, **no shared database bottleneck**.
// Flush thread wakes immediately and drains the accumulated queue.
375
-
376
-
// ── After: check for any rows that exhausted retries ────────────────
377
-
if (db.backupDeadLetterCount>0) {
378
-
console.warn(`${db.backupDeadLetterCount} rows need manual replay`);
379
-
// Run the bundled CLI to replay them:
380
-
// ./arkilian-dlq app.sqlite --replay
381
-
}
382
-
383
-
// ── Detect CDC gap (outbox was full during the incident) ─────────────
384
-
if (db.capturePaused) {
385
-
console.warn('A capture gap occurred — verify the hourly snapshot covered it');
386
-
// capturePaused clears automatically after the next successful GCS upload.
387
-
}
388
318
389
319
db.close();
390
320
```
321
+
```
391
322
392
323
## System Constraints and Design Choices
393
324
Unlike complex distributed SQLite systems (e.g., LiteFS or rqlite), Arkilian embraces single-writer architectures partitioned by micro-datasets. It purposefully avoids:
0 commit comments