ADR for append-only change log in mint database#2173
Open
crodas wants to merge 4 commits into
Open
Conversation
crodas
force-pushed
the
proposal/append-only
branch
from
a324543 to
d17ce03
Compare
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #2173 +/- ##
==========================================
- Coverage 73.56% 73.56% -0.01%
==========================================
Files 360 360
Lines 81432 81432
==========================================
- Hits 59909 59906 -3
- Misses 21523 21526 +3 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
asmogo
reviewed
Jul 1, 2026
Comment on lines
+114
to
+122
| ### `change_id` | ||
|
|
||
| Application-generated `i64`: | ||
|
|
||
| ``` | ||
| Bit 63: always 0 (positive signed i64) | ||
| Bits 62..22: 41 bits of millisecond timestamp since custom epoch | ||
| Bits 21..0: 22 bits of hash(changeset) | ||
| ``` |
Collaborator
There was a problem hiding this comment.
can we use an auto-increment or sequence here? It would simplify the logic and fix the duplicate key conflicts.
Collaborator
Author
There was a problem hiding this comment.
If we do, the database service assigning the autoincrement number would serialize all our operations. That is not a problem with SQLite, but it would be with Postgres.
Comment on lines
+70
to
+75
| change_id BIGINT PRIMARY KEY, | ||
| quote_id TEXT NOT NULL, | ||
| state TEXT, | ||
| request_lookup_id TEXT, | ||
| payment_proof TEXT, | ||
| paid_time BIGINT |
Collaborator
There was a problem hiding this comment.
i think this is missing some member variables (request_lookup_id_kind gets updated in update_melt_quote_request_lookup_id or estimated_blocks, fee_reserve in update_melt_quote_state )
Collaborator
Author
There was a problem hiding this comment.
Good catch, anything that is updated should be reflected here.
| | Entity | Reason | | ||
| |--------|--------| | ||
| | **mint_quote** | Already append-only. `update_mint_quote` writes to `mint_quote_payments` and `mint_quote_issued` detail tables. The `amount_paid`/`amount_issued` on the quote row are materialized sums. | | ||
| | **blind_signature** | Insert-only, never updated or deleted. | |
Collaborator
There was a problem hiding this comment.
there are certain conditions in add_blind_signatures that result in a update statement for the blind_signature.
crodas
force-pushed
the
proposal/append-only
branch
from
549235f to
d88bfb3
Compare
3 tasks
crodas
force-pushed
the
proposal/append-only
branch
from
d88bfb3 to
5201ef0
Compare
Capture the design decision behind the append-only branch so the reasoning is recorded alongside the code. The mint database overwrites or deletes rows on state changes, which makes auditing, replay, and reconciliation hard. The ADR proposes per-table typed log tables to preserve history without changing existing read/write paths, and works across SQLite and PostgreSQL. Also adds the ADR template.
The append-only change log ADR previously proposed per-table log tables and kept the single delta table only as an alternative. We are pursuing the delta table instead: one insert-only table, one insert path, and one forward-compatible Delta enum. A new tracked field becomes a new enum variant rather than a schema migration and a new _log table. Changes: - Rewrite the decision and design around a single change_log table (id, record, delta, reason, created_at) and demote per-table log tables to a rejected alternative alongside the generic event_log and triggers. - Derive the Delta enum from the mutation method signatures, one variant per writable field, so the set of possible changes is defined in code. - Generate id as an in-house Snowflake i64 (timestamp, node id, sequence) with no external crate; keep created_at as its own column so readers do not have to decode the id. - Add a serialization-format comparison (JSON, CBOR, protobuf, postcard/bincode, hand-rolled) and recommend CBOR via ciborium, already used for token encoding: self-describing, forward compatible, compact. - Document using the table as an ordered, durable event stream consumed by a cursor over id, a poor man's Kafka for replication, cache invalidation, and cross-instance sync, including the pre-commit id visibility race and its retention coupling to the slowest consumer.
The ADR described an earlier delta-only change_log design that the prototype/append-only reference implementation had since moved past. Update the proposal so it can be evaluated against real code, and fold in a schema refinement discovered while reviewing it. Rewrite the design to match the reference implementation: a journal table storing Event = Snapshot | Delta so current state replays from an empty database, JSON serialization (cashu types do not round-trip through CBOR), and mint-orchestrated emission via a JournalTransaction append primitive rather than piggy-backed SQL-layer inserts. Propose a compound (entity, record) key in place of the concatenated "table_name:pk" string, with entity stored as a small Entity enum discriminant. This avoids repeating the table name as text per row, makes per-entity queries an indexed range scan, and removes delimiter-parsing ambiguity. Everything remains a proposal; the branch is referenced as the reference implementation.
The reference implementation on prototype/append-only moved past what the ADR described. Several "the reference currently does X" statements had gone stale, so anyone reading the ADR to evaluate the design would be comparing against a version of the code that no longer exists. Correct the drift: the (entity, record) compound key and Entity enum already landed in the reference, so stop framing them as an unimplemented proposal. Document the JournaledDatabase decorator that now drives emission (replacing the scattered mint-layer call sites), fix the add_journal snippet to the two-arg form, and note that no production read/replay path exists yet, which makes the event-stream section a design sketch.
crodas
force-pushed
the
proposal/append-only
branch
from
5201ef0 to
abb0034
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Proposal solution for append-only storage: here is the rendered. Feedback and inline comments are more than welcome
Here is the rendered document
Once this proposal is approved and merged I will open a PR.
Notes to the reviewers
Suggested CHANGELOG Updates
CHANGED
ADDED
REMOVED
FIXED
Checklist
just quick-checkbefore committingcrates/cdk-ffi)