Skip to content

Commit 8272bd6

Browse files
0xpatrickdev0xpatrickbot
authored andcommitted
docs(designs): correct git-backbone crux to GC, not axis-3 transport
The load-bearing motivation is reachability-driven GC: a refs/formulas/<id> <-> sqlite formula-graph mirror so git's own GC collects what no live formula roots. Axes 3 (transport) and 2 (trees) are extensions, not the crux. Add a Status section recording the shipped axes 1+4 substrate (git2/libgit2, bare repo at {dir}/cas.git) and the JS/Rust seam blocking the formula-graph mirror. Sync designs/README.md.
1 parent bf1e4a6 commit 8272bd6

2 files changed

Lines changed: 60 additions & 17 deletions

File tree

designs/README.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -174,7 +174,7 @@ LLM-agent stack).*
174174
| [daemon-git-remotes](daemon-git-remotes.md) | 2026-05-18 | 2026-05-29 | Proposed |
175175
| [daemon-git-next-steps](daemon-git-next-steps.md) | 2026-05-27 | 2026-06-03 | Proposed |
176176
| [endo-fs-from-git](endo-fs-from-git.md) | 2026-05-28 | 2026-05-28 | In Progress |
177-
| [daemon-git-backbone](daemon-git-backbone.md) | 2026-05-27 | 2026-05-28 | Proposed |
177+
| [daemon-git-backbone](daemon-git-backbone.md) | 2026-05-27 | 2026-05-29 | In Progress |
178178
| [daemon-message-streaming](daemon-message-streaming.md) | 2026-03-26 | 2026-05-19 | In Progress (PR #287) |
179179
| [daemon-mount](daemon-mount.md) | 2026-03-20 | 2026-05-27 | In Progress |
180180
| [daemon-mount-capabilities](daemon-mount-capabilities.md) | 2026-05-18 | 2026-05-27 | **Complete** |
@@ -597,7 +597,7 @@ capabilities available to agents.
597597
| daemon-git-capability | Proposed | Revised git design over `EndoMount` / `EndoMountEntry`; `tree(ref)` and `readOnly()` both live on the `Git` cap |
598598
| daemon-git-remotes | Proposed | MVP remote-git companion: fetch / pull / push composed from local `Git`, bounded HTTPS transport, endpoint policy, and credential caps |
599599
| daemon-git-next-steps | Proposed | The version-controlled filesystem loop milestone over the canonical trio: north-star agent loop (provide workspace → read/list/edit → status/diff → commit → pull/push → inspect history via `filesystemAt(ref)`) and the content/versioning/network/historical-read/bulk-storage layer split. Open `- [ ]` work: worked bot-fork reference flow, `provideGitClone` + identity boundary (→ `daemon-git-clone.md`), `tree(ref)`/`filesystemAt(ref)` reconciliation. Agent-tools layer deferred to #416 |
600-
| daemon-git-backbone | Proposed | Back the existing Rust CAS (`rust/endo/src/cas.rs`) with git, all four axes: objects → git object DB (sha256-keyed), `TreeManifest` → git trees, bulk transport off CapTP onto pack/smart-protocol, retention → `refs/formulas/<id>` + `git gc`. In-process `gix` (recommended) in the `endor` supervisor; sha256 stays the content key behind a sha256→oid index. Migrations out of scope. Four axes; each probe-able separately |
600+
| daemon-git-backbone | In Progress | Back the existing Rust CAS (`rust/endo/src/cas.rs`) with git, all four axes: objects → git object DB (sha256-keyed), `TreeManifest` → git trees, bulk transport off CapTP onto pack/smart-protocol, retention → `refs/formulas/<id>` + git GC. The crux is GC: a `refs/formulas/<id>` ↔ sqlite formula-graph mirror so git's own GC collects what no live formula roots. In-process `git2` (libgit2, vendored) in the `endor` supervisor; bare repo at `{dir}/cas.git`; sha256 stays the content key behind a sha256→oid index. Axes 1+4-substrate shipped on PR #369; the formula-graph mirror is blocked on a JS/Rust seam (formula graph lives in the JS daemon, not reachable from the Rust CAS). Migrations out of scope. Four axes; each probe-able separately |
601601
| filesystem-watchers | Not Started | `EndoMount.followNameChanges` parity with `EndoDirectory`; Node `fs.watch` adapter on `FilePowers` |
602602
| daemon-locator-terminology | Not Started | Clean locator API; unblocked |
603603
| daemon-rename-to-manager | Not Started | Rename `daemon.js`/`Daemon`/`MignonicPowers` to `manager.js`/`Manager`/`WorkerPowers` to align JS with Rust `endor` nomenclature |

designs/daemon-git-backbone.md

Lines changed: 58 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -3,9 +3,43 @@
33
| | |
44
|---|---|
55
| **Created** | 2026-05-27 |
6-
| **Updated** | 2026-05-28 |
6+
| **Updated** | 2026-05-29 |
77
| **Author** | 0xPatrick (prompted) |
8-
| **Status** | Proposed |
8+
| **Status** | In Progress |
9+
10+
## Status
11+
12+
Axes 1 and 4 are built on PR
13+
[#369](https://github.com/endojs/endo-but-for-bots/pull/369) (`endo`
14+
crate).
15+
The git library is **`git2` (libgit2 bindings)**, not the `gix`
16+
recommendation in Open Question 1 / § The Rust lever below — the
17+
maintainer ratified the mature, broadly-covered library; libgit2 is
18+
vendored (bundled C, no system dependency), so the no-C rationale for
19+
`gix` was moot because the `endo` crate already links bundled C
20+
(rusqlite's SQLite, xsnap's XS).
21+
The substrate is a **bare git repository at `{dir}/cas.git`**: objects
22+
at `{dir}/cas.git/objects/`, retention refs at
23+
`{dir}/cas.git/refs/cas/<sha256>`, with the `sha256 → git-oid`
24+
resolution index kept flat at `{dir}` (Open Question 2's sha256 content
25+
key behind a SHA-1 git object DB).
26+
GC is an in-process manual reachability sweep (libgit2 ships no
27+
`git gc` porcelain) seeded from the `refs/cas/*` refs.
28+
29+
**Axis 4 is not yet complete.** Its highest-value piece — the
30+
`refs/formulas/<id>` ↔ sqlite formula-graph mirror that makes GC
31+
reachability-driven *from formula liveness* — is not built. The shipped
32+
`refs/cas/<sha256>` refs are keyed by content hash, not formula id, and
33+
nothing connects the formula graph to retain/release calls; the `cas-gc`
34+
call sites still pass an empty caller-root set (now harmless only
35+
because the durable `refs/cas/*` refs seed the live set). The mirror is
36+
blocked on an architecture seam: the formula graph lives in the **JS**
37+
daemon (`packages/daemon/src/daemon-database.js`, the SQLite `formula`
38+
table), mutated by JS (`daemon.js``persistencePowers.writeFormula` /
39+
`deleteFormula`); the Rust supervisor sees only opaque SQL strings
40+
crossing the XS FFI and cannot observe formula liveness. The mirror wire
41+
must live in JS, and no JS code calls the `cas-*` verbs today. See the
42+
journal seam report (2026-05-29) for the recommended landing site.
943

1044
## Summary
1145

@@ -25,14 +59,20 @@ This design replaces the hand-rolled substrate underneath the existing
2559
git** — while keeping the daemon's content identity, its CapTP control
2660
plane, and the `cas-*` verb surface unchanged.
2761

28-
The load-bearing motivation is **axis 3**: get bulk data *out of the
29-
CapTP data plane*.
30-
Whole-tree reads, archive ingestion, and cross-peer content sync are
31-
the wrong shape for one-object-per-CapTP-turn; git's pack/smart-protocol
32-
is purpose-built for exactly that traffic.
33-
The other three axes are the substrate that makes axis 3 mechanical:
34-
once blobs and trees *are* git objects (axis 1, axis 2) and retention
35-
*is* refs (axis 4), shipping them over the git wire is free.
62+
The load-bearing motivation is **GC** — axis 4, plus the axis-1
63+
substrate it needs: a `refs/formulas/<id>` ↔ sqlite-formula-graph mirror
64+
so git itself can do the garbage collection.
65+
The retention roots are the formula graph; the most straightforward way
66+
to maintain the relationship is for git to hold a ref per live formula
67+
identifier, kept in sync with the formula rows, and to let git's own GC
68+
collect everything no ref keeps reachable.
69+
The other axes are extensions, not the crux: getting bulk data *out of
70+
the CapTP data plane* (axis 3 — whole-tree reads, archive ingestion,
71+
cross-peer content sync, the wrong shape for one-object-per-CapTP-turn)
72+
and native git trees (axis 2) are valuable payoffs that the same git
73+
substrate makes mechanical, but the reason to adopt git at all is the
74+
crash-safe, reachability-driven live set the hand-rolled CAS does not
75+
have.
3676

3777
The four axes map almost 1:1 onto the existing CAS's four phases:
3878

@@ -252,7 +292,8 @@ not today.
252292
| `.meta` `refs` count | superseded by refs (axis 4) |
253293

254294
The flat `{dir}/{hex-sha256}` directory and `.meta` sidecars disappear;
255-
`{statePath}/store/.git/objects/` holds every blob and tree.
295+
the bare git repository at `{dir}/cas.git/objects/` holds every blob
296+
and tree.
256297
The `type` advisory in `.meta`
257298
([daemon-cas-management](daemon-cas-management.md) § Content types)
258299
folds into git's native type tag, except for the daemon-specific
@@ -290,7 +331,7 @@ Two properties improve on the swap (see § What surprised us):
290331
Axis 2 is mechanical once axis 1 lands; it has no new external
291332
contract.
292333

293-
## Axis 3 — bulk transport off the CapTP data plane (load-bearing)
334+
## Axis 3 — bulk transport off the CapTP data plane (extension)
294335

295336
CapTP / OCapN stays the **control plane**: capability handshakes,
296337
eventual sends, retention subscriptions, GC negotiation
@@ -320,9 +361,11 @@ sideband (see Open Question 4).
320361
is **unchanged** — small token-streamed text is exactly what CapTP is
321362
for; axis 3 targets only bulk transfers.
322363

323-
This is the axis the whole design is *for*: axes 1/2/4 are the
324-
substrate that makes the data plane git-shaped, and axis 3 is the
325-
payoff of getting bulk bytes out of the capability envelopes.
364+
This is a high-value extension once the substrate is git-shaped: axes
365+
1/2/4 make the data plane git-shaped, and axis 3 is the payoff of
366+
getting bulk bytes out of the capability envelopes. It is not the crux
367+
(that is axis 4's reachability-driven GC); it is the largest payoff the
368+
crux's substrate unlocks.
326369

327370
## Axis 4 — retention becomes `refs/formulas/<id>` + `git gc`
328371

0 commit comments

Comments
 (0)