DCPerf: drop corrupt first perf interval from AMD report - #751
Open
charles-typ wants to merge 1 commit into
Open
DCPerf: drop corrupt first perf interval from AMD report#751charles-typ wants to merge 1 commit into
charles-typ wants to merge 1 commit into
Conversation
Summary: On Turin (AMD EPYC 9D25 / Zen5), DCPerf `sleepbench` `overall-metrics.csv` reported a DRAM Read BW of ~620 GB/s — above the part's 512 GB/s physical ceiling. Root cause: the first `perf stat -I` interval is corrupt. The uncore/DF memory-controller counters report a cumulative startup value on their first read, and the first interval's duration is often irregular, so its derived rate (~2,454 GB/s here, ~4.8x the max) is physically impossible. Nothing dropped or clamped that first sample before it was averaged into the summary. Fix in `perfutils/generate_amd_perf_report.py`: add `drop_first_interval()`, called in `main()` after metrics are built and before aggregation/timeseries emission. It drops the first interval (`num_sockets` rows, one per socket) from every metric series so the summary and the timeseries stay aligned and physically meaningful. Short runs (series <= `num_sockets`) are left untouched to avoid collapsing to empty. Excluding the first sample, sustained DRAM read ~252 GB/s (valid). [Session trajectory link](https://www.internalfb.com/intern/devai/devmate/inspector/?id=8ff846bd-428a-4c33-8fa7-3c6264d4c96a) Differential Revision: D110798470
meta-cla
Bot
added
the
CLA Signed
|
@charles-typ has exported this pull request. If you are a Meta employee, you can view the originating Diff in D110798470. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary:
On Turin , DCPerf
sleepbenchoverall-metrics.csvreported a DRAM Read BW of ~620 GB/s — above the part's 512 GB/s physical ceiling.Root cause: the first
perf stat -Iinterval is corrupt. The uncore/DF memory-controller counters report a cumulative startup value on their first read, and the first interval's duration is often irregular, so its derived rate (~2,454 GB/s here, ~4.8x the max) is physically impossible. Nothing dropped or clamped that first sample before it was averaged into the summary.Fix in
perfutils/generate_amd_perf_report.py: adddrop_first_interval(), called inmain()after metrics are built and before aggregation/timeseries emission. It drops the first interval (num_socketsrows, one per socket) from every metric series so the summary and the timeseries stay aligned and physically meaningful. Short runs (series <=num_sockets) are left untouched to avoid collapsing to empty.Excluding the first sample, sustained DRAM read ~252 GB/s (valid).
Differential Revision: D110798470