Skip to content

Commit a9f631d

Browse files
NickCirvclaude
andcommitted
fix(bench): keep 'reduction' framing — green CI after effective-metric rename
The effective-saving metric I just added used 'saving' wording and renamed the md row the honesty regression test pins ('Aggregate per-file structural reduction'), turning CI red (1073/1074, 20cb061). Use 'reduction' throughout (effectiveReduction / 'EFFECTIVE reduction' / 'Effective structural reduction'), restore the pinned row label. Honest framing preserved + the effective metric intact. Gate: tsc clean; honesty-claims 5/5; full suite 1074/1074 (verified BEFORE commit this time). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
1 parent 20cb061 commit a9f631d

1 file changed

Lines changed: 7 additions & 7 deletions

File tree

bench/real-world.ts

Lines changed: 7 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -165,7 +165,7 @@ async function main(): Promise<void> {
165165
// Size-guarded: engram never costs more than the raw read (the Read hook
166166
// passes a file through when its packet would be larger), so the EFFECTIVE
167167
// cost is min(packet, file). This is what engram actually delivers in a
168-
// real session — the honest saving, never negative on small files.
168+
// real session — the honest reduction, never negative on small files.
169169
let totalEffectiveEngram = 0;
170170

171171
for (const abs of files) {
@@ -209,7 +209,7 @@ async function main(): Promise<void> {
209209
totalBaseline > 0 ? ((totalBaseline - totalEngram) / totalBaseline) * 100 : 0;
210210
// The honest, shipped-behaviour number: engram only intercepts when its
211211
// packet is smaller, so on a real session it never adds tokens.
212-
const effectiveSavings =
212+
const effectiveReduction =
213213
totalBaseline > 0
214214
? ((totalBaseline - totalEffectiveEngram) / totalBaseline) * 100
215215
: 0;
@@ -239,10 +239,10 @@ async function main(): Promise<void> {
239239
);
240240
console.log();
241241
console.log(
242-
`EFFECTIVE saving (what engram actually delivers — it only intercepts files`
242+
`EFFECTIVE reduction (what engram actually delivers — it only intercepts`
243243
);
244244
console.log(
245-
`it shrinks; the rest read normally and engram adds nothing): ${effectiveSavings.toFixed(
245+
`files it shrinks; the rest read normally and engram adds nothing): ${effectiveReduction.toFixed(
246246
1
247247
)}%`
248248
);
@@ -303,7 +303,7 @@ async function main(): Promise<void> {
303303
totalBaselineTokens: totalBaseline,
304304
totalEngramTokens: totalEngram,
305305
savingsPct: Number(aggregateSavings.toFixed(2)),
306-
effectiveSavingsPct: Number(effectiveSavings.toFixed(2)),
306+
effectiveReductionPct: Number(effectiveReduction.toFixed(2)),
307307
effectiveEngramTokens: totalEffectiveEngram,
308308
interceptedFiles,
309309
wins,
@@ -325,8 +325,8 @@ async function main(): Promise<void> {
325325
`|---|---|`,
326326
`| Baseline tokens (all files, raw Read, uncached) | **${totalBaseline.toLocaleString()}** |`,
327327
`| engramx tokens (rich packets) | **${totalEngram.toLocaleString()}** |`,
328-
`| Raw per-file structural reduction | ${aggregateSavings.toFixed(1)}% |`,
329-
`| **Effective saving (size-guarded — what engram actually delivers)** | **${effectiveSavings.toFixed(1)}%** |`,
328+
`| Aggregate per-file structural reduction (raw) | ${aggregateSavings.toFixed(1)}% |`,
329+
`| **Effective structural reduction (size-guarded — what engram delivers)** | **${effectiveReduction.toFixed(1)}%** |`,
330330
`| Files engram intercepts (packet smaller) | ${interceptedFiles} of ${perFile.length} |`,
331331
`| Median per-file reduction | ${median.toFixed(1)}% |`,
332332
`| Files where the packet was smaller | ${wins} of ${perFile.length} |`,

0 commit comments

Comments
 (0)