🐛 Harden encrypted notebook history and index recovery #19180

Author: 88250

docs/ENCRYPTED-NOTEBOOK.md

@@ -210,11 +210,11 @@ File history:
   auto-generated before close/exit → ensures history is captured even after locking
   history index → content left empty (ciphertext not indexed for search)
   view history → read the snapshot boxID, object type, and stable file basename, then reconstruct the original AAD for decryption (notebook must be unlocked)
-  roll back → decrypt with the snapshot AAD → verify that the root-block ID matches the basename → load tree → restore ciphertext verbatim when the basename is unchanged, or let WriteTree re-envelope it when the basename changes
+  roll back → decrypt with the snapshot AAD → verify that the root-block ID matches the basename → load tree → let WriteTree serialize the restored content and re-envelope it with the target basename
 
 Deleted-notebook history:
   before deletion → entire directory backed up as ciphertext to history dir
-  restore → verify WrappedDEK with the matching global key backup and master password, then copy the ciphertext directory back verbatim
+  restore → copy the ciphertext directory back verbatim without opening it; authenticate WrappedDEK with the matching global key backup and master password when unlocking
   key retention → while kernel-managed history remains recoverable, deletion of the global key backup it depends on is forbidden; permanently purging the history removes the dependency
 
 database history:
@@ -223,7 +223,7 @@ database history:
   roll back → encrypted-notebook database file/resources rolled back to notebook-level dir
 ```
 
-Parent directories below the history root are not part of AAD. Every history entry retains, in an unambiguous canonical form, the `boxID`, object type, and stable file basename or object identifier at snapshot creation; these values may be encoded by a fixed history layout or stored as metadata committed with the snapshot. Changing only a live document's parent directory leaves its AAD context unchanged; if its basename changes, existing history continues to authenticate with the old basename. Rollback first verifies the ciphertext and that the basename matches the decrypted object ID, then restores under the target parent directory. Re-enveloping is required only when the target basename changes. Missing, malformed, or mismatched snapshot context rejects viewing and rollback; the implementation must not parse ciphertext as plaintext or replace the snapshot basename with the current basename.
+Parent directories below the history root are not part of AAD. Every history entry retains, in an unambiguous canonical form, the `boxID`, object type, and stable file basename or object identifier at snapshot creation; these values may be encoded by a fixed history layout or stored as metadata committed with the snapshot. Changing only a live document's parent directory leaves its AAD context unchanged; if its basename changes, existing history continues to authenticate with the old basename. Rollback first verifies the ciphertext and that the basename matches the decrypted object ID, then restores under the target parent directory. Document rollback also normalizes the tree and resolves duplicate block IDs, so WriteTree serializes the restored content and creates a fresh envelope even when the basename is unchanged. The historical ciphertext remains intact. Copying a deleted notebook back is a closed-state restoration, not proof that the current key backup matches; authentication happens on unlock, and a missing matching backup may be supplied later. Missing, malformed, or mismatched snapshot context rejects viewing and rollback; the implementation must not parse ciphertext as plaintext or replace the snapshot basename with the current basename.
 
 ## 9. Block-Ref Cross-Boundary Protection
 
@@ -374,10 +374,10 @@ The global configuration lock is acquired before any notebook lifecycle lock. An
 
 Persistent formats are layered; global KDF configuration, key envelopes, and per-object data envelopes are not conflated:
 
-1. **Data-object envelope**: Every `.sy`, asset, and database-definition object contains at least a format marker or version, an algorithm identifier, a cryptographically random nonce unique for the same purpose key, ciphertext, and an authentication tag. The current format constructs AAD deterministically as `format domain + boxID + object type + stable object ID`, excluding parent directories and absolute paths. A `.sy` stable object ID is its canonical basename `<rootID>.sy`; an asset uses its desensitized disk basename, a database definition uses its avID, and a fixed singleton file uses a versioned constant ID. An asset's original name is authenticated encrypted metadata in the same asset container. After decryption, the internal object ID matches the AAD object ID, and duplicate object IDs within one notebook are rejected. AES-GCM nonces come from the operating system CSPRNG; randomness failure aborts the write, and the standard per-purpose-key invocation bound is enforced. An authentication tag cannot retrospectively identify nonce reuse, so the design never claims such reuse will automatically cause authentication failure.
+1. **Data-object envelope**: Every `.sy`, asset, and database-definition object contains at least a format marker or version, an algorithm identifier, a 96-bit nonce generated by the operating system CSPRNG, ciphertext, and an authentication tag. The current format constructs AAD deterministically as `format domain + boxID + object type + stable object ID`, excluding parent directories and absolute paths. A `.sy` stable object ID is its canonical basename `<rootID>.sy`; an asset uses its desensitized disk basename, a database definition uses its avID, and a fixed singleton file uses a versioned constant ID. An asset's original name is authenticated encrypted metadata in the same asset container. After decryption, the internal object ID matches the AAD object ID, and duplicate object IDs within one notebook are rejected. Randomness failure aborts the write. Nonce uniqueness is probabilistic: the current implementation has no persistent per-purpose-key invocation counter shared across restarts, devices, and restored backups, and therefore does not enforce a lifetime invocation bound. This is a limitation, not a guarantee of unlimited safe use. Adding such enforcement requires a versioned multi-device key-lifecycle and recovery design that preserves access to existing data. An authentication tag cannot retrospectively identify nonce reuse, so the design never claims such reuse will automatically cause authentication failure.
 2. **Global key and envelope metadata**: `NotebookCrypto` stores the KDF algorithm and parameters, MasterSalt, verifier format, KEK-envelope version, and backup-HMAC version; `BoxConf.WrappedDEK` stores each notebook's DEK envelope. They are not duplicated in every data object.
 3. **Purpose separation**: Each notebook's DEK derives subkeys with fixed domain separators for file content, assets, database definitions, content SQLCipher, and blocktree SQLCipher. Asset-name metadata and content chunks share the asset subkey and are separated by distinct AAD suffixes; a separate subkey for every object type is not required. The KEK may be used directly for the verifier, DEK wrapping, and backup HMAC, with those uses separated by different algorithms, versioned AAD, or authenticated data formats. WrappedDEK authentication is not incorrectly placed in a DEK domain.
-4. **Database compatibility metadata**: SQLCipher version, cipher parameters, and schema version live in a verifiable database header or the corresponding encrypted configuration boundary, not every data object. Content and blocktree indexes are rebuildable; after source ciphertext authenticates, an incompatible index is closed and rebuilt rather than weakening cipher parameters or falling back to plaintext SQLite.
+4. **Database compatibility metadata**: SQLCipher version, cipher parameters, index kind, and schema version are stored in the SQLCipher-authenticated encrypted_index_meta table and checked before the index is reused. Content and blocktree indexes are rebuildable; an index without compatibility metadata, with incompatible settings, or that fails to open is closed; after the notebook key and metadata and all source documents authenticate, both indexes may be recreated and populated by mounting. Failure to authenticate a source document preserves the source and residual indexes and returns an error. Rebuilding never weakens cipher parameters or falls back to plaintext SQLite.
 5. **Rename and format stability**: The current document format constructs AAD from the stable object ID and is already frozen as a compatibility baseline. Existing unversioned `SYAE` assets remain readable without requiring users to delete or recreate attachments when upgrading to version 2. Writes use version 2; reads do not force migration of files or historical snapshots. A parent-only move does not re-envelope content. A stable-basename change authenticates and decrypts with the old basename and creates a new envelope with the new basename. Future changes to envelopes, AAD semantics, or key derivation require an explicit format version and authenticated legacy reads, or a recoverable migration before the legacy reader is removed. Migration authenticates source data first and preserves the original data and matching recovery material so failure or interruption remains recoverable; resetting MasterSalt, discarding keys, or deleting data must never resolve format incompatibility. Unknown formats, corruption, and authentication failures preserve the original files and return an error without falling back to plaintext or bypassing authentication. Every format change requires fixtures from the supported previous format to verify affected read, export, history, backup, and recovery paths.
 
 The SQLCipher main database, WAL, SHM, rollback journal, temporary files, and backup copies are protected objects. They remain in controlled directories and use their corresponding purpose key. Locking, crash recovery, and history snapshots enumerate these companion files; unencrypted SQLite pages, query results, or diagnostics must never be written to a global temporary directory.
@@ -427,12 +427,12 @@ Native mobile save flows must call `AcquireExportFile` to obtain a lease contain
 | Refs, moves, mirrors, assets, and database operations between an encrypted notebook and a normal notebook or another encrypted notebook | Island boundaries hold; rejection creates no partial files, global indexes, or relation data |
 | Encrypted asset-container write fails | The caller receives an error, no separate name mapping or plaintext temporary file is created, and an incomplete target file cannot be read as a valid asset |
 | Delete an encrypted notebook while retaining history, then disable or restore | A key dependency prevents deletion of global config and backup; the matching backup and master password restore the notebook, and only explicit permanent purge removes the dependency |
-| Move a document's parent directory or change its basename after creating history, then view or restore the old entry | Authenticate and decrypt with the snapshot boxID, object type, and stable basename recorded by the history entry; rollback re-envelopes only when the target basename changes, while missing snapshot context rejects the operation |
+| Move a document's parent directory or change its basename after creating history, then view or restore the old entry | Authenticate and decrypt with the snapshot boxID, object type, and stable basename recorded by the history entry; rollback serializes the restored tree and re-envelopes it with the target basename while preserving the historical ciphertext, while missing snapshot context rejects the operation |
 | Disk inspection of files, assets, databases, WAL/SHM, history, snapshots, and logs | No readable body, asset contents, original asset names, or plaintext SQLite pages exist in encrypted-notebook persistent locations; `temp/` may contain plaintext during export or after cleanup failure |
 | Restart after an abnormal exit | Startup removes at least entries under `temp/export/` whose first-level name is a valid boxID; generic temporary-directory initialization may also remove other exports and plugin temporary files |
 | Ciphertext tampering, path substitution, and backup corruption | Authentication or format validation fails safely; never fall back to a normal path, generate replacement key material, or silently overwrite configuration |
 | Both an encrypted notebook's `conf.json` and notebook key backup are missing, or sync removes an encrypted notebook that is currently open locally | Ciphertext or runtime identity puts the former in `Error` without creating a normal configuration; the latter drains in-flight leases and clears the DEK, mount, dedicated databases, caches, and indexes even though `Conf.GetBox` is empty after deletion |
-| Nonce randomness failure, reaching a purpose-key invocation bound, or discovering nonce reuse afterwards | Reject before encryption on randomness failure or bound exhaustion; AES-GCM authentication cannot detect nonce reuse that already occurred, so discovery is handled as a key-compromise incident by rotating the DEK and re-encrypting data |
+| Nonce randomness failure or discovering nonce reuse afterwards | Reject before encryption on randomness failure; no persistent lifetime invocation bound is currently enforced. AES-GCM authentication cannot detect nonce reuse that already occurred; discovery requires migration to a new DEK, not merely changing the master password |
 | Replay an authenticated historical ciphertext object or key backup | It may validate as an authentic old version but is never claimed to be freshest or from a trusted source; rollback risk is explicit, and the current design does not claim rollback prevention |
 | Master-password change and KEK rewrapping | WrappedDEKs and backup update atomically; an interrupted update leaves the old configuration recoverably usable; the old-password exposure warning is accurate; data and history are verifiably readable before and after migration |
 | Concurrent multi-notebook operations and auto-lock | Lock order has no deadlock; authenticated UI activity or an explicit keepalive refreshes all currently unlocked notebooks, while background work does not refresh them automatically; a locking notebook admits no new lease |