Skip to content

Commit 37407e8

Browse files
committed
docs: remove control plane architecture details and add links to arkilian.com for keys/config
1 parent da824d1 commit 37407e8

1 file changed

Lines changed: 28 additions & 97 deletions

File tree

README.md

Lines changed: 28 additions & 97 deletions
Original file line numberDiff line numberDiff line change
@@ -202,17 +202,12 @@ is back to the exact state it left off — including every write that shipped
202202
while the old instance was live.
203203

204204
```js
205-
// server.js — Cloud Run / Fly.io / any container
205+
// server.js
206206
import Arkilian from 'arkilian';
207207

208-
const TENANT = process.env.TENANT_ID; // e.g. "acme-corp"
209-
const API_KEY = process.env.ARKILIAN_DATABASE_TOKEN;
210-
211-
// Cold-start restore: download the latest snapshot + replay incremental WAL.
212-
// No-op if the local file is already up-to-date.
213-
Arkilian.hydrate(`${TENANT}.sqlite`, 'https://api.arkilian.com', API_KEY);
214-
215-
const db = new Arkilian(API_KEY, `${TENANT}.sqlite`);
208+
// Get your API token from https://arkilian.com
209+
const API_TOKEN = process.env.ARKILIAN_DATABASE_TOKEN;
210+
const db = new Arkilian(API_TOKEN, 'app.sqlite');
216211

217212
// Schema is auto-created; capture triggers are wired automatically.
218213
db.exec(`CREATE TABLE IF NOT EXISTS orders (
@@ -222,13 +217,13 @@ db.exec(`CREATE TABLE IF NOT EXISTS orders (
222217
ts INTEGER NOT NULL DEFAULT (unixepoch())
223218
)`);
224219

225-
// Every INSERT is captured and shipped to the control plane in < 2 s.
220+
// Every INSERT is captured and shipped in < 2 s.
226221
export function placeOrder(item, qty) {
227222
db.run('INSERT INTO orders (item, qty) VALUES (?, ?)', [item, qty]);
228223
return db.lastInsertRowid;
229224
}
230225

231-
// Health endpoint — wire to Cloud Run liveness probe.
226+
// Health endpoint.
232227
export function health() {
233228
return {
234229
healthy: db.backupHealthy,
@@ -241,66 +236,30 @@ export function health() {
241236
process.on('SIGTERM', () => db.close());
242237
```
243238

244-
**What happens on GCP:**
245-
- Deploy to **Cloud Run** (scales to zero — Arkilian's 2-second poll loop costs nothing at idle).
246-
- Point `ARKILIAN_SIGNED_URL_ENDPOINT` at a **Cloud Function** that issues GCS signed URLs.
247-
- Every hourly snapshot lands in a **Cloud Storage** bucket (`gs://arkilian-backups/<tenant>/`).
248-
- The WAL stream feeds your control plane, which fans out to **BigQuery** for analytics.
249-
250239
---
251240

252-
### 2 — Real-time analytics pipeline into BigQuery
241+
### 2 — Real-time CDC Pipeline
253242

254-
Arkilian's WAL push endpoint ships every row change as replayable SQL within
255-
2 seconds of commit. Wire your control plane to publish those payloads onto
256-
**Pub/Sub** and let a Dataflow pipeline hydrate **BigQuery** in near-real time.
243+
Configure the background worker with your `ARKILIAN_DATABASE_TOKEN` and endpoints obtained from [arkilian.com](https://arkilian.com) to stream raw row operations in real time.
257244

258245
```js
259-
// Control-plane webhook handler (Cloud Functions / Cloud Run)
260-
// POST /v1/wal/push — called by Arkilian's flush thread
261-
import { BigQuery } from '@google-cloud/bigquery';
262-
263-
const bq = new BigQuery();
264-
const dataset = bq.dataset('arkilian_cdc');
265-
266-
export async function walPushHandler(req, res) {
267-
const { db_id, payload_id, sql, params } = req.body;
268-
269-
// Idempotency: Arkilian guarantees at-least-once; dedupe on payload_id.
270-
await dataset.table('raw_events').insert([{
271-
db_id,
272-
payload_id,
273-
sql,
274-
params: JSON.stringify(params),
275-
received_at: BigQuery.datetime(new Date().toISOString()),
276-
}], { skipInvalidRows: false, ignoreUnknownValues: false });
277-
278-
res.status(200).json({ ok: true });
279-
}
280-
```
246+
import Arkilian from 'arkilian';
281247

282-
```sql
283-
-- BigQuery scheduled query: materialize the orders table from CDC
284-
SELECT
285-
JSON_VALUE(params, '$[0]') AS item,
286-
CAST(JSON_VALUE(params, '$[1]') AS INT64) AS qty,
287-
received_at
288-
FROM `project.arkilian_cdc.raw_events`
289-
WHERE sql LIKE 'INSERT INTO orders%'
290-
ORDER BY received_at;
291-
```
248+
// Get your configuration and API token from https://arkilian.com
249+
const token = process.env.ARKILIAN_DATABASE_TOKEN;
250+
const db = new Arkilian(token, 'app.sqlite');
292251

293-
The result: **sub-5-second latency** from SQLite write to BigQuery row — without
294-
Kafka, Debezium, or a managed database. The entire pipeline is SQLite on the
295-
edge, a Cloud Function in the middle, and BigQuery at the end.
252+
db.exec(`CREATE TABLE IF NOT EXISTS users (
253+
id INTEGER PRIMARY KEY,
254+
email TEXT NOT NULL UNIQUE
255+
)`);
256+
```
296257

297258
---
298259

299-
### 3 — Offline-first mobile backend (Go / Cloud Run)
260+
### 3 — Offline-first Go Backend
300261

301-
The Go binding lets you embed Arkilian directly into a Go service with zero CGO
302-
overhead beyond the initial open. Here a game server persists per-player state
303-
locally and replicates automatically.
262+
Link the native library directly into your Go binaries. Acquire the client SDK assets and environment configuration templates from [arkilian.com](https://arkilian.com).
304263

305264
```go
306265
// main.go
@@ -320,9 +279,10 @@ import (
320279

321280
func main() {
322281
var db *C.arkilian
323-
path := C.CString("players.sqlite")
282+
path := C.CString("app.sqlite")
324283
defer C.free(unsafe.Pointer(path))
325284

285+
// Initialize using configuration obtained from https://arkilian.com
326286
if C.db_init(&db, path) != 0 {
327287
log.Fatal("db_init failed")
328288
}
@@ -335,59 +295,30 @@ func main() {
335295
)`)
336296
defer C.free(unsafe.Pointer(sql))
337297
C.db_exec(db, sql)
338-
339-
// Every score update ships to GCS within 2 seconds.
340-
upd := C.CString("UPDATE players SET score = score + 1 WHERE id = 1")
341-
defer C.free(unsafe.Pointer(upd))
342-
C.db_exec(db, upd)
343-
344-
// Monitoring
345-
fmt.Printf("queue=%d healthy=%d\n",
346-
C.db_backup_queue_depth(db),
347-
C.db_backup_is_healthy(db))
348298
}
349299
```
350300

351-
**Deploy pattern on GCP:**
352-
- Build to a Docker image → push to **Artifact Registry** → run on **Cloud Run** or **GKE**.
353-
- Each player shard is a Cloud Run instance with its own `players.sqlite`.
354-
- Arkilian replicates to **Cloud Storage**; a Cloud Scheduler job triggers hydration on instance spin-up.
355-
- 5,000 shards = 5,000 independent SQLite files, each replicating at 2-second cadence, **no shared database bottleneck**.
356-
357301
---
358302

359-
### 4 — Incident response: kill-switch & dead-letter replay
303+
### 4 — Incident Response: Kill-Switch & Diagnostics
304+
305+
Manage backups dynamically without restarting the application process.
360306

361307
```js
362308
import Arkilian from 'arkilian';
363309

310+
// Retrieve your API token from https://arkilian.com
364311
const db = new Arkilian(process.env.ARKILIAN_DATABASE_TOKEN, 'app.sqlite');
365312

366-
// ── Incident: upstream destination is down ──────────────────────────
367-
// Stop shipping without losing any captured rows.
368-
// Rows continue queuing in _pending_backup; nothing is dropped.
313+
// Pause all outbound backup traffic instantly during an upstream outage.
369314
db.setBackupEnabled(false);
370-
console.log('Backup paused. Queue depth:', db.backupQueueDepth);
371315

372-
// ── Mitigation resolved: resume ─────────────────────────────────────
316+
// Resume normal operations.
373317
db.setBackupEnabled(true);
374-
// Flush thread wakes immediately and drains the accumulated queue.
375-
376-
// ── After: check for any rows that exhausted retries ────────────────
377-
if (db.backupDeadLetterCount > 0) {
378-
console.warn(`${db.backupDeadLetterCount} rows need manual replay`);
379-
// Run the bundled CLI to replay them:
380-
// ./arkilian-dlq app.sqlite --replay
381-
}
382-
383-
// ── Detect CDC gap (outbox was full during the incident) ─────────────
384-
if (db.capturePaused) {
385-
console.warn('A capture gap occurred — verify the hourly snapshot covered it');
386-
// capturePaused clears automatically after the next successful GCS upload.
387-
}
388318

389319
db.close();
390320
```
321+
```
391322
392323
## System Constraints and Design Choices
393324
Unlike complex distributed SQLite systems (e.g., LiteFS or rqlite), Arkilian embraces single-writer architectures partitioned by micro-datasets. It purposefully avoids:

0 commit comments

Comments
 (0)