feat(mint): add a replayable append-only journal for mint state#2218
Draft
crodas wants to merge 2 commits into
Draft
feat(mint): add a replayable append-only journal for mint state#2218crodas wants to merge 2 commits into
crodas wants to merge 2 commits into
Conversation
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## main #2218 +/- ##
==========================================
+ Coverage 73.56% 73.73% +0.16%
==========================================
Files 360 366 +6
Lines 81432 82044 +612
==========================================
+ Hits 59909 60492 +583
- Misses 21523 21552 +29 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
crodas
force-pushed
the
prototype/append-only
branch
from
30155cd to
9d8118b
Compare
Collaborator
The mint kept current state only in mutable tables, so every update erased the previous value and every delete dropped the row. Auditing, replay, and reconciliation of financially meaningful transitions were impossible after the fact. Add a single insert-only `journal` table that is replayable on its own: from an empty database a consumer loads each entity's creation snapshot and applies its field-level deltas in id order to reconstruct current state. The same table doubles as an ordered event stream keyed by a time-sortable id. Event model (cdk-common `database::event_log`): - `Event` is either a `Snapshot`, the full base object captured at creation (mint quote, melt quote, proof, blind signature, keyset), or a `Delta`, one writable field's new value. `From` conversions let call sites write `value.into()` instead of spelling out the wrapping. - Events serialize as JSON, the encoding the mint already uses for these types. A binary format like CBOR is more compact but several cashu types do not round-trip through a non-human-readable serializer. - `id` is an in-house lock-free Snowflake i64 (millisecond timestamp, node id, per-millisecond sequence) that is monotonic, time-sortable, and unique across concurrent writers and, given a node id per instance, across mint instances. Compound (entity, record) key: - A row is identified by a typed `Entity` discriminant plus the row's bare primary key, not a concatenated `"table_name:pk"` string. `Entity` has explicit `#[repr(u8)]` discriminants, `as_u8`, and a `TryFrom<u8>` that rejects unknown values; the writer derives it from the event via `Event::entity()`, so an event and its entity can never disagree. Schema (sqlite + postgres migrations): - Add the insert-only `journal` table with columns `id`, `entity`, `record`, `event`, `created_at`, indexed by `(entity, record, id)` so both a single row's history and a whole entity type are indexed lookups. Orchestration (mint layer + signatory): - Journaling is a `JournalTransaction::add_journal` supertrait method shared by the main write transaction and the keyset transaction. The SQL layer provides only the durable append, running in the caller's transaction so it commits or rolls back with the mutation it records. The mint decides which events to emit and wires the issue, melt, swap, rollback, and compensation flows; the signatory emits keyset creation and activation on rotation. Serde support: - Derive serde on the snapshotted domain types (`MintQuote`, `IncomingPayment`, `Issuance`), reusing the existing amount-currency helper and adding `amount_currency_serde_opt` for the optional amount field. Tests: - Flow-driven tests drive a real issue, swap, and melt and assert the emitted journal rows replay to the expected state. A cdk-sqlite test replays the keyset and melt-quote lifecycles and cross-checks every stored entity against its event. Unit tests cover the id generator, the event-to-entity mapping, and the discriminant round-trip; a `read_journal` helper reads rows back through rusqlite.
The append-only journal was populated by hand: every state mutation in the mint orchestration was paired with an explicit add_journal call in the same transaction. Nothing tied a mutation to its journal write, so a new or edited mutation path could silently produce an incomplete journal, and one production site already did (the signatory's boot-time keyset re-activation in init_keysets emitted no event). Auditing and replay are only as trustworthy as the journal is complete, so completeness must be structural, not a convention that each call site is trusted to follow. Make journaling a property of the storage layer. JournaledDatabase<D> wraps a mint database and, for every mutation of a journaled entity (mint quote, melt quote, proof, blind signature, keyset), appends the matching event on the same transaction as the mutation. The append runs on the inner transaction's own connection, so the journal row commits or rolls back atomically with the state it records. The decorator lives in cdk-common, so a single implementation journals for every backend (sqlite, postgres, supabase) instead of duplicating the logic in each store. Generic over an unsized D held behind an Arc, the wrapper accepts either a concrete database or an existing trait object. Wrapping the DynMintDatabase handle the mint already threads around lets journaling be installed at two construction choke points, Mint::new_internal and DbSignatory::new, rather than at every backend or test. All mints and signatories pass through those, so the decorator is the only way to open a transaction. The orchestration layer no longer calls add_journal anywhere: the manual pairs in the issue, melt, swap, and rollback flows and the signatory keyset flow are removed, along with the now-redundant pre-read that fetched the superseded keyset. add_journal remains on the transaction trait as the internal primitive the decorator invokes on the inner transaction, but the decorator's own add_journal, on both the main and keyset transactions, rejects direct calls with a new Error::JournalNotPermitted. Because the decorator's mutation methods journal through the inner transaction and never through themselves, that rejection is unreachable internally and fires only when code outside the decorator tries to write the journal directly, which would produce an unmanaged, possibly duplicate row. Two mutations carry more than a plain column value, so the decorator derives their events rather than copying an argument: - update_mint_quote drains a change buffer (take_changes), so the wrapper peeks the pending payments and issuances through a new non-draining accessor on MintQuote before delegating, then journals each from a clone once the persist succeeds. This keeps the per-payment and per-issuance deltas, which are richer than the amount_paid and amount_issued totals the row stores. - set_active_keyset deactivates the keyset it supersedes, whose identity is not one of its arguments. The keyset wrapper captures the active-keyset map when the transaction opens and journals the deactivation of the previous keyset from it. Wrapping the signatory keystore before init_keysets means the boot-time re-activation is now journaled too, closing the pre-existing gap. Serde and the event vocabulary are unchanged; the decorator emits the same events the hand-written calls did, verified by the existing flow and replay tests passing without modification. New tests drive two keyset rotations plus a reboot, asserting the supersede deactivation and the boot re-activation are journaled, and assert that a direct add_journal on either wrapped transaction is rejected. A non-draining pending_changes accessor is added to MintQuote for the change-buffer peek.
crodas
force-pushed
the
prototype/append-only
branch
from
813a898 to
47b3898
Compare
Collaborator
Author
|
@thesimplekid, this is a draft PR, not meant for reviews but to illustrate #2173. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Reference implementation for #2173
Notes to the reviewers
Suggested CHANGELOG Updates
CHANGED
ADDED
REMOVED
FIXED
Checklist
just quick-checkbefore committingcrates/cdk-ffi)