-
Notifications
You must be signed in to change notification settings - Fork 173
bug: --full flag silently ignored during benchmark — per-transaction .csv.zst never written #476
Description
Bug: --full flag silently ignored — per-transaction .csv.zst file never written
Description
Running any benchmark with the --full flag does not produce a per-transaction
.csv.zst output file. Only the .json.zst aggregate is written, making post-hoc
per-operation analysis impossible. The flag appears to be accepted without error but
has no effect.
Steps to Reproduce
warp put --host=<host> --access-key=<key> --secret-key=<secret> --fullExpected: Two output files written after the benchmark:
warp-put-<timestamp>.csv.zst— one row per operation (full per-transaction log)warp-put-<timestamp>.json.zst— aggregated summary
Actual: Only warp-put-<timestamp>.json.zst is written. The .csv.zst file is
never created.
Root Cause
Introduced in commit 483f844, which added addCollector(). That function was
written to gate per-transaction collection on ctx.Bool("full"), but the --full
flag was only registered in analyzeFlags — never added to benchFlags. As a
result, ctx.Bool("full") always returns false during benchmark execution, and
the collector always falls back to EmptyOpsCollector (no ops stored, no file
written).
// addCollector() — intended gate:
if ctx.Bool("full") { // always false during benchmark — flag not registered
// fan ops to NewOpsCollector + LiveCollector
} else {
return bench.EmptyOpsCollector, updates // always taken
}Impact
warp analyze --full <file>(full per-operation latency/throughput analysis) is
effectively unreachable — there is no.csv.zstfile to analyze.- Users explicitly requesting
--fullfor detailed post-benchmark analysis get
silently degraded output with no warning or error. warp merge,warp cmp, and per-request filtering (--analyze.op,
--analyze.host,--analyze.skip) all require a.csv.zst— none of these
workflows are usable.
Additional Finding
Running warp analyze file.csv.zst without --full re-aggregates individual
operations into 1-second buckets before reporting — producing results equivalent to
reading the .json.zst aggregate — with no indication to the user that per-operation
precision has been lost.
Fix
A fix is available in PR #475, which:
- Makes
addCollector()correctly gate on--full(flag already present in
analyzeFlags, which is combined withbenchFlagsfor all benchmark commands). - Makes
--fulladditive: both.csv.zstand.json.zstare always written
together when the flag is set. - Mirrors the fix in
benchserver.gofor distributed mode. - Adds a warning in
warp analyzewhen a.csv.zstis analyzed without--full,
directing users to the more accurate analysis path. - Updates README documentation and adds 7 unit tests.
PR: #475