Skip to content

Commit 68453f6

Browse files
jlucaso1jlucaso1
committed
fix(bench/signal): give each summary its own session pair
The original bench shared one Alice/Bob pair across Encrypt, Decrypt, and Round-trip benches. This was always borderline-correct, but became broken once encrypt got fast enough for mitata to enable batch mode: - mitata's min_cpu_time = 642ms / encrypt cost - Pre-migration: ~169µs/iter → ~3800 iterations during the Encrypt bench - Post-migration: ~17µs/iter (batch-amortised) → ~38000 iterations When the Encrypt bench advances Alice's chain solo past 25_000 messages without Bob seeing them, the next Decrypt bench (Alice encrypts → Bob decrypts) has Bob trying to skip > MAX_FORWARD_JUMPS (25_000) messages forward in the chain, which libsignal correctly rejects as InvalidMessage. Fix: each `summary()` block builds a fresh (Alice, Bob) pair via make{Wasm,Lib}Pair() helpers. The Encrypt bench can advance its own Alice's chain freely; the Decrypt and Round-trip benches operate on independent pairs that start fresh. Verified: 5/5 runs of `node --expose-gc benches/signal.ts` pass with consistent timings (~17µs encrypt, 30-250µs decrypt, ~800µs round-trip on the Rust WASM side). Reproducer for the underlying limit: for (let i = 0; i < 25500; i++) await alice.encrypt(typical); const enc = await alice.encrypt(typical); await bob.decryptWhisperMessage(enc.body); // throws InvalidMessage The threshold is exactly libsignal's MAX_FORWARD_JUMPS = 25_000.
1 parent 9fddfba commit 68453f6

1 file changed

Lines changed: 140 additions & 153 deletions

File tree

‎benches/signal.ts‎

Lines changed: 140 additions & 153 deletions
Original file line numberDiff line numberDiff line change
@@ -95,114 +95,104 @@ class LibsignalStore implements SignalStorage {
9595
}
9696

9797
// ============================================================================
98-
// Setup for Rust WASM benchmarks
98+
// Setup helpers — each bench group gets its own fresh session pair so the
99+
// ratchet doesn't drift past libsignal's MAX_FORWARD_JUMPS (25_000) when one
100+
// bench advances the chain solo (e.g. an Encrypt-only bench) before Decrypt
101+
// has a chance to catch up.
99102
// ============================================================================
100-
const aliceStorage = new FakeStorage();
101-
const bobStorage = new FakeStorage();
102103

103-
const wasmBobAddress = new ProtocolAddress("bob", 1);
104-
const wasmAliceAddress = new ProtocolAddress("alice", 1);
105-
106-
// Trust each other's identities
107-
aliceStorage.trustIdentity("bob", bobStorage.ourIdentityKeyPair.pubKey);
108-
bobStorage.trustIdentity("alice", aliceStorage.ourIdentityKeyPair.pubKey);
109-
110-
// Generate Bob's pre-keys for WASM
111-
const bobSignedPreKeyId = 1;
112-
const bobSignedPreKey = generateSignedPreKey(
113-
bobStorage.ourIdentityKeyPair,
114-
bobSignedPreKeyId,
115-
);
116-
const bobOneTimePreKey = generatePreKey(100);
117-
118-
bobStorage.storeSignedPreKey(bobSignedPreKey.keyId, bobSignedPreKey);
119-
bobStorage.storePreKey(bobOneTimePreKey.keyId, bobOneTimePreKey.keyPair);
120-
121-
const bobBundle = {
122-
registrationId: bobStorage.ourRegistrationId,
123-
identityKey: bobStorage.ourIdentityKeyPair.pubKey,
124-
signedPreKey: {
125-
keyId: bobSignedPreKey.keyId,
126-
publicKey: bobSignedPreKey.keyPair.pubKey,
127-
signature: bobSignedPreKey.signature,
128-
},
129-
preKey: {
130-
keyId: bobOneTimePreKey.keyId,
131-
publicKey: bobOneTimePreKey.keyPair.pubKey,
132-
},
104+
type WasmPair = {
105+
alice: SessionCipher;
106+
bob: SessionCipher;
133107
};
134108

135-
// Alice processes Bob's bundle and establishes session
136-
const aliceSessionBuilder = new SessionBuilder(aliceStorage, wasmBobAddress);
137-
await aliceSessionBuilder.processPreKeyBundle(bobBundle);
138-
139-
const aliceCipher = new SessionCipher(aliceStorage, wasmBobAddress);
140-
const bobCipher = new SessionCipher(bobStorage, wasmAliceAddress);
109+
async function makeWasmPair(): Promise<WasmPair> {
110+
const aliceStorage = new FakeStorage();
111+
const bobStorage = new FakeStorage();
112+
const bobAddr = new ProtocolAddress("bob", 1);
113+
const aliceAddr = new ProtocolAddress("alice", 1);
114+
115+
aliceStorage.trustIdentity("bob", bobStorage.ourIdentityKeyPair.pubKey);
116+
bobStorage.trustIdentity("alice", aliceStorage.ourIdentityKeyPair.pubKey);
117+
118+
const sk = generateSignedPreKey(bobStorage.ourIdentityKeyPair, 1);
119+
const pk = generatePreKey(100);
120+
bobStorage.storeSignedPreKey(sk.keyId, sk);
121+
bobStorage.storePreKey(pk.keyId, pk.keyPair);
122+
123+
await new SessionBuilder(aliceStorage, bobAddr).processPreKeyBundle({
124+
registrationId: bobStorage.ourRegistrationId,
125+
identityKey: bobStorage.ourIdentityKeyPair.pubKey,
126+
signedPreKey: {
127+
keyId: sk.keyId,
128+
publicKey: sk.keyPair.pubKey,
129+
signature: sk.signature,
130+
},
131+
preKey: { keyId: pk.keyId, publicKey: pk.keyPair.pubKey },
132+
});
141133

142-
// ============================================================================
143-
// Setup for libsignal-node benchmarks
144-
// ============================================================================
145-
const aliceLibsignalStorage = new LibsignalStore();
146-
const bobLibsignalStorage = new LibsignalStore();
134+
const alice = new SessionCipher(aliceStorage, bobAddr);
135+
const bob = new SessionCipher(bobStorage, aliceAddr);
147136

148-
const libsignalBobAddress = new libsignalNode.ProtocolAddress("bob", 1);
149-
const libsignalAliceAddress = new libsignalNode.ProtocolAddress("alice", 1);
137+
// PreKey handshake + reply so both sides have established sessions
138+
const first = await alice.encrypt(Buffer.from("hi"));
139+
await bob.decryptPreKeyWhisperMessage(first.body);
140+
const reply = await bob.encrypt(Buffer.from("ack"));
141+
await alice.decryptWhisperMessage(reply.body);
150142

151-
// Trust each other's identities
152-
aliceLibsignalStorage.trustIdentity(
153-
"bob",
154-
Buffer.from(bobLibsignalStorage.ourIdentityKeyPair.pubKey),
155-
);
156-
bobLibsignalStorage.trustIdentity(
157-
"alice",
158-
Buffer.from(aliceLibsignalStorage.ourIdentityKeyPair.pubKey),
159-
);
143+
return { alice, bob };
144+
}
160145

161-
// Generate Bob's pre-keys for libsignal-node
162-
const bobLibSignedPreKey = libsignalKeyHelper.generateSignedPreKey(
163-
bobLibsignalStorage.ourIdentityKeyPair,
164-
bobSignedPreKeyId,
165-
);
166-
const bobLibOneTimePreKey = libsignalKeyHelper.generatePreKey(100);
146+
type LibPair = {
147+
alice: InstanceType<typeof libsignalNode.SessionCipher>;
148+
bob: InstanceType<typeof libsignalNode.SessionCipher>;
149+
};
167150

168-
bobLibsignalStorage.storeSignedPreKey(
169-
bobLibSignedPreKey.keyId,
170-
bobLibSignedPreKey,
171-
);
172-
bobLibsignalStorage.storePreKey(
173-
bobLibOneTimePreKey.keyId,
174-
bobLibOneTimePreKey.keyPair,
175-
);
151+
async function makeLibPair(): Promise<LibPair> {
152+
const aliceStorage = new LibsignalStore();
153+
const bobStorage = new LibsignalStore();
154+
const bobAddr = new libsignalNode.ProtocolAddress("bob", 1);
155+
const aliceAddr = new libsignalNode.ProtocolAddress("alice", 1);
156+
157+
aliceStorage.trustIdentity(
158+
"bob",
159+
Buffer.from(bobStorage.ourIdentityKeyPair.pubKey),
160+
);
161+
bobStorage.trustIdentity(
162+
"alice",
163+
Buffer.from(aliceStorage.ourIdentityKeyPair.pubKey),
164+
);
165+
166+
const sk = libsignalKeyHelper.generateSignedPreKey(
167+
bobStorage.ourIdentityKeyPair,
168+
1,
169+
);
170+
const pk = libsignalKeyHelper.generatePreKey(100);
171+
bobStorage.storeSignedPreKey(sk.keyId, sk);
172+
bobStorage.storePreKey(pk.keyId, pk.keyPair);
173+
174+
const builder = new libsignalNode.SessionBuilder(aliceStorage, bobAddr);
175+
await builder.initOutgoing({
176+
registrationId: bobStorage.ourRegistrationId,
177+
identityKey: bobStorage.ourIdentityKeyPair.pubKey,
178+
signedPreKey: {
179+
keyId: sk.keyId,
180+
publicKey: sk.keyPair.pubKey,
181+
signature: sk.signature,
182+
},
183+
preKey: { keyId: pk.keyId, publicKey: pk.keyPair.pubKey },
184+
});
176185

177-
const bobLibsignalBundle = {
178-
registrationId: bobLibsignalStorage.ourRegistrationId,
179-
identityKey: bobLibsignalStorage.ourIdentityKeyPair.pubKey,
180-
signedPreKey: {
181-
keyId: bobLibSignedPreKey.keyId,
182-
publicKey: bobLibSignedPreKey.keyPair.pubKey,
183-
signature: bobLibSignedPreKey.signature,
184-
},
185-
preKey: {
186-
keyId: bobLibOneTimePreKey.keyId,
187-
publicKey: bobLibOneTimePreKey.keyPair.pubKey,
188-
},
189-
};
186+
const alice = new libsignalNode.SessionCipher(aliceStorage, bobAddr);
187+
const bob = new libsignalNode.SessionCipher(bobStorage, aliceAddr);
190188

191-
// Alice processes Bob's bundle and establishes session
192-
const aliceLibsignalBuilder = new libsignalNode.SessionBuilder(
193-
aliceLibsignalStorage,
194-
libsignalBobAddress,
195-
);
196-
await aliceLibsignalBuilder.initOutgoing(bobLibsignalBundle);
189+
const first = await alice.encrypt(Buffer.from("hi"));
190+
await bob.decryptPreKeyWhisperMessage(first.body as unknown as Uint8Array);
191+
const reply = await bob.encrypt(Buffer.from("ack"));
192+
await alice.decryptWhisperMessage(reply.body as unknown as Uint8Array);
197193

198-
const aliceLibsignalCipher = new libsignalNode.SessionCipher(
199-
aliceLibsignalStorage,
200-
libsignalBobAddress,
201-
);
202-
const bobLibsignalCipher = new libsignalNode.SessionCipher(
203-
bobLibsignalStorage,
204-
libsignalAliceAddress,
205-
);
194+
return { alice, bob };
195+
}
206196

207197
// ============================================================================
208198
// Test Messages
@@ -215,95 +205,92 @@ const typicalMessage = Buffer.from(
215205
),
216206
);
217207

218-
// ============================================================================
219-
// Establish bidirectional sessions (required for decrypt benchmarks)
220-
// ============================================================================
221-
222-
// WASM: Alice sends first message (PreKeyWhisperMessage)
223-
const wasmFirstEncrypted = await aliceCipher.encrypt(typicalMessage);
224-
// Bob decrypts first message - this establishes Bob's session with Alice
225-
await bobCipher.decryptPreKeyWhisperMessage(wasmFirstEncrypted.body);
226-
// Bob replies to Alice - this completes the ratchet setup
227-
const wasmBobReply = await bobCipher.encrypt(typicalMessage);
228-
// Alice decrypts Bob's reply - now both sides have established sessions
229-
await aliceCipher.decryptWhisperMessage(wasmBobReply.body);
230-
231-
// libsignal-node: Same pattern
232-
const libsignalFirstEncrypted =
233-
await aliceLibsignalCipher.encrypt(typicalMessage);
234-
await bobLibsignalCipher.decryptPreKeyWhisperMessage(
235-
libsignalFirstEncrypted.body as unknown as Uint8Array,
236-
);
237-
const libsignalBobReply = await bobLibsignalCipher.encrypt(typicalMessage);
238-
await aliceLibsignalCipher.decryptWhisperMessage(
239-
libsignalBobReply.body as unknown as Uint8Array,
240-
);
241-
242-
// Now both Alice and Bob have fully established sessions in both libraries
208+
// Each bench group below gets its own fresh session pair. This prevents one
209+
// bench (e.g. Encrypt) from advancing the ratchet past the
210+
// MAX_FORWARD_JUMPS=25_000 threshold and breaking subsequent Decrypt benches.
243211

244212
// ============================================================================
245-
// Benchmarks
213+
// Encrypt benchmarks
246214
// ============================================================================
215+
const encWasm = await makeWasmPair();
216+
const encLib = await makeLibPair();
247217

248218
boxplot(() => {
249219
summary(() => {
250220
bench("Encrypt typical message (Rust WASM)", async () => {
251-
const result = await aliceCipher.encrypt(typicalMessage);
221+
const result = await encWasm.alice.encrypt(typicalMessage);
252222
do_not_optimize(result);
253223
}).gc("inner");
254224

255225
bench("Encrypt typical message (libsignal-node)", async () => {
256-
const result = await aliceLibsignalCipher.encrypt(typicalMessage);
226+
const result = await encLib.alice.encrypt(typicalMessage);
257227
do_not_optimize(result);
258228
}).gc("inner");
259229
});
230+
});
231+
232+
// ============================================================================
233+
// Decrypt benchmarks (fresh sessions — independent of Encrypt above)
234+
// ============================================================================
235+
const decWasm = await makeWasmPair();
236+
const decLib = await makeLibPair();
260237

238+
// Use mitata's generator form so the encrypt setup runs outside the timed
239+
// section (the [0] callback is invoked per-iteration before t0). The bench
240+
// body only measures Bob's decrypt. Each iteration consumes a fresh
241+
// ciphertext because both ratchets advance on every encrypt/decrypt.
242+
boxplot(() => {
261243
summary(() => {
262-
// Decrypt benchmarks - Alice encrypts, Bob decrypts
263-
// Both sides now have established sessions, so these are WhisperMessages (type 1)
264-
bench("Decrypt WhisperMessage (Rust WASM)", async () => {
265-
// Alice encrypts a fresh message to Bob
266-
const encrypted = await aliceCipher.encrypt(typicalMessage);
267-
// Bob decrypts it - this advances the ratchet
268-
const result = await bobCipher.decryptWhisperMessage(encrypted.body);
269-
do_not_optimize(result);
244+
bench("Decrypt WhisperMessage (Rust WASM)", function* () {
245+
yield {
246+
[0]: async () => await decWasm.alice.encrypt(typicalMessage),
247+
bench: async (encrypted: { body: Uint8Array }) => {
248+
const result = await decWasm.bob.decryptWhisperMessage(encrypted.body);
249+
do_not_optimize(result);
250+
},
251+
};
270252
}).gc("inner");
271253

272-
bench("Decrypt WhisperMessage (libsignal-node)", async () => {
273-
// Alice encrypts a fresh message to Bob
274-
const encrypted = await aliceLibsignalCipher.encrypt(typicalMessage);
275-
// Bob decrypts it - this advances the ratchet
276-
const result = await bobLibsignalCipher.decryptWhisperMessage(
277-
encrypted.body as unknown as Uint8Array,
278-
);
279-
do_not_optimize(result);
254+
bench("Decrypt WhisperMessage (libsignal-node)", function* () {
255+
yield {
256+
[0]: async () => await decLib.alice.encrypt(typicalMessage),
257+
bench: async (encrypted: { body: unknown }) => {
258+
const result = await decLib.bob.decryptWhisperMessage(
259+
encrypted.body as unknown as Uint8Array,
260+
);
261+
do_not_optimize(result);
262+
},
263+
};
280264
}).gc("inner");
281265
});
266+
});
282267

268+
// ============================================================================
269+
// Full round-trip benchmarks (fresh sessions)
270+
// ============================================================================
271+
const rtWasm = await makeWasmPair();
272+
const rtLib = await makeLibPair();
273+
274+
boxplot(() => {
283275
summary(() => {
284-
// Full round-trip: Alice -> Bob -> Alice
285276
bench("Full round-trip encrypt+decrypt (Rust WASM)", async () => {
286-
// Alice sends to Bob
287-
const toBob = await aliceCipher.encrypt(typicalMessage);
288-
const decryptedByBob = await bobCipher.decryptWhisperMessage(toBob.body);
289-
// Bob replies to Alice
290-
const toAlice = await bobCipher.encrypt(typicalMessage);
291-
const decryptedByAlice = await aliceCipher.decryptWhisperMessage(
277+
const toBob = await rtWasm.alice.encrypt(typicalMessage);
278+
const decryptedByBob = await rtWasm.bob.decryptWhisperMessage(toBob.body);
279+
const toAlice = await rtWasm.bob.encrypt(typicalMessage);
280+
const decryptedByAlice = await rtWasm.alice.decryptWhisperMessage(
292281
toAlice.body,
293282
);
294283
do_not_optimize(decryptedByBob);
295284
do_not_optimize(decryptedByAlice);
296285
}).gc("inner");
297286

298287
bench("Full round-trip encrypt+decrypt (libsignal-node)", async () => {
299-
// Alice sends to Bob
300-
const toBob = await aliceLibsignalCipher.encrypt(typicalMessage);
301-
const decryptedByBob = await bobLibsignalCipher.decryptWhisperMessage(
288+
const toBob = await rtLib.alice.encrypt(typicalMessage);
289+
const decryptedByBob = await rtLib.bob.decryptWhisperMessage(
302290
toBob.body as unknown as Uint8Array,
303291
);
304-
// Bob replies to Alice
305-
const toAlice = await bobLibsignalCipher.encrypt(typicalMessage);
306-
const decryptedByAlice = await aliceLibsignalCipher.decryptWhisperMessage(
292+
const toAlice = await rtLib.bob.encrypt(typicalMessage);
293+
const decryptedByAlice = await rtLib.alice.decryptWhisperMessage(
307294
toAlice.body as unknown as Uint8Array,
308295
);
309296
do_not_optimize(decryptedByBob);

0 commit comments

Comments
 (0)