Commit 70d281a
Wire breakdown.csv into feedsim_autoscale_dlrm + add preprocessing/postprocessing logs (#736)
Summary:
Pull Request resolved: #736
Three related changes so perfpub can identify the actual benchmarking phase and report metrics filtered to that window (instead of averaging across warmup + drain + idle periods).
1. jobs.yml: add `benchmarks/feedsim/breakdown.csv` to the copymove `after:` list for feedsim_autoscale_dlrm. The other feedsim jobs (search-mode, mini, etc.) already had it; only the DLRM autoscale variant was missing. With this, perfpub's automatic breakdown-based metric filtering kicks in for `./perfpub --no-xdb --no-manifold --dir benchmark_metrics_<id>` (see perfpub/README.md "Automatic Metric Filtering with breakdown.csv").
2. run.sh: add preprocessing/postprocessing log entries around the existing search_qps.sh main_benchmark window. search_qps.sh already logs main_benchmark start/end around `sleep $experiment_time` (the actual measurement window); this commit adds the surrounding phase info so the full timeline is visible in breakdown.csv:
- preprocessing = run.sh start → driver launch (server bring-up, graph/model load, warmup)
- main_benchmark = experiment_time window (unchanged)
- postprocessing = driver exit → run.sh exit (queue drain, leaf shutdown)
perfpub uses only main_benchmark for default filtering; pre/post are informational for humans inspecting the CSV.
3. runtime_breakdown_utils.sh: make `create_breakdown_csv` idempotent (skip if file exists). Without this, multi-instance feedsim runs (run-feedsim-multi.sh launching N instances of run.sh) race on truncating the CSV — the second instance's create_breakdown_csv call would silently drop the first instance's already-logged entries. Verified t20 q=100/q=120 runs now record both inst1+inst2 preprocessing/main_benchmark/postprocessing entries cleanly. benchpress's copymove hook (is_move: true) moves the file out between iterations, so stale data across runs is not a concern.
Verified on rtptest3440 BGM (qps=100, 120 each at 180s/300s):
- breakdown.csv lands in benchmark_metrics_<id>/ after each run
- `./perfpub --no-xdb --no-manifold --dir benchmark_metrics_<id>` succeeds and emits overall-metrics.csv with the breakdown-filtered window
- All 4 expected operation entries present per instance: preprocessing start/end, main_benchmark start/end, postprocessing start/end (no race-induced drops)
Reviewed By: YifanYuan3
Differential Revision: D105756766
fbshipit-source-id: 7ce888f9d8e962a0c255c16992ead4ece23879a51 parent e013cca commit 70d281a
3 files changed
Lines changed: 22 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
781 | 781 | | |
782 | 782 | | |
783 | 783 | | |
| 784 | + | |
784 | 785 | | |
785 | 786 | | |
786 | 787 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
25 | 25 | | |
26 | 26 | | |
27 | 27 | | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
28 | 38 | | |
29 | 39 | | |
30 | 40 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
574 | 574 | | |
575 | 575 | | |
576 | 576 | | |
| 577 | + | |
| 578 | + | |
| 579 | + | |
| 580 | + | |
| 581 | + | |
| 582 | + | |
577 | 583 | | |
578 | 584 | | |
579 | 585 | | |
| |||
822 | 828 | | |
823 | 829 | | |
824 | 830 | | |
| 831 | + | |
| 832 | + | |
| 833 | + | |
825 | 834 | | |
826 | 835 | | |
827 | 836 | | |
| |||
876 | 885 | | |
877 | 886 | | |
878 | 887 | | |
| 888 | + | |
879 | 889 | | |
880 | 890 | | |
| 891 | + | |
881 | 892 | | |
882 | 893 | | |
883 | 894 | | |
| |||
0 commit comments