Report native CMN mesh (uncore) performance metrics - #605
Open
marziehlenjaniMeta wants to merge 1 commit into
Open
Report native CMN mesh (uncore) performance metrics#605marziehlenjaniMeta wants to merge 1 commit into
marziehlenjaniMeta wants to merge 1 commit into
Conversation
meta-cla
Bot
added
the
CLA Signed
|
@marziehlenjaniMeta has exported this pull request. If you are a Meta employee, you can view the originating Diff in D102244354. |
meta-codesync
Bot
changed the title
marziehlenjaniMeta
force-pushed
the
export-D102244354-to-v2-beta
branch
from
July 9, 2026 06:07
07465d8 to
99fe390
Compare
marziehlenjaniMeta
pushed a commit
to marziehlenjaniMeta/DCPerf
that referenced
this pull request
Jul 9, 2026
…earch#605) Summary: Rename CMN Memory Read Bandwidth to CMN Memory Bandwidth since the underlying CMN event hns_mc_reqs_local_all counts both read and write requests to the memory controller. CMN-Cypress does not distinguish read vs write MC requests in this event, so the previous "Read" label was misleading. Differential Revision: D102244354
meta-codesync
Bot
changed the title
marziehlenjaniMeta
pushed a commit
to marziehlenjaniMeta/DCPerf
that referenced
this pull request
Jul 9, 2026
…#605) Summary: **Problem** The Neoverse V3 perf report exposed a single CMN memory metric ("CMN Memory Read Bandwidth") derived from `hns_mc_reqs_local_all`. That name was misleading (the event counts read+write) and, more importantly, the metric was incomplete: it ignored remote (cross-chiplet) memory traffic and surfaced none of the other CMN mesh telemetry the hardware exposes. An earlier attempt to add DRAM read/write bandwidth via the external Arm cmn-tools repo (port-filtered XP watchpoints, cloned at install time) produced unusable numbers on Phoenix — write bandwidth reported as 0 and mesh bandwidth of ~680 GB/s — and added network/`/dev/mem`/kernel-module fragility to the install path. **Why** Root cause of the cmn-tools garbage: Meta fleet telemetry (dynolog) holds ~1.3k CMN perf_event fds continuously, starving the CMN DTC's small hardware counter budget. cmn-tools' watchpoint programming collides with the arm-cmn driver under that contention. Named perf events do not — perf multiplexes and scales them, so steady-state values stay accurate. Separately, the Cypress HN-S MC-request event genuinely has no read/write split (confirmed against the event set and the Phoenix SoC Architecture Specification, Table 15-4), so a DRAM byte-level read/write split is not observable from HN-S counters at all. **Fix** Drop the cmn-tools apparatus entirely (install_platform_tools.sh, cmn_mesh_discovery.sh, cmn_topdown_runner.sh, the install.py / topdown.py hooks, and the log-parsing path in the report generator). Instead report native CMN PMU metrics mapped to the "HNF Effectiveness" group in the Phoenix SoC Architecture Specification, Table 15-4: - **CMN Memory Bandwidth (MBps)** — read+write total, now summing local **and** remote MC requests across both chiplets (each 64B request counted once at its home HN-S, so no double-counting). - **CMN Local / Remote Memory Bandwidth (MBps)** — NUMA locality split. - **CMN Memory Read Mix %** — read share via HN-S QoS PoCQ occupancy (the only read/write proxy the HN-S exposes). - **CMN MC Retry %** — memory-controller backpressure (Table 15-4 mc_retry). - **CMN Snoop Filter Hit Rate %** — coherence-directory effectiveness (sf_hit_ratio). - **CMN Mesh Frequency (GHz)** — from dtc_cycles, for DVFS-throttling visibility. The collect script keeps the CMN event group intentionally lean because of the dynolog counter contention described above. Differential Revision: D102244354
marziehlenjaniMeta
force-pushed
the
export-D102244354-to-v2-beta
branch
from
July 9, 2026 06:08
99fe390 to
91818e3
Compare
marziehlenjaniMeta
pushed a commit
to marziehlenjaniMeta/DCPerf
that referenced
this pull request
Jul 9, 2026
…#605) Summary: **Problem** The Neoverse V3 perf report exposed a single CMN memory metric ("CMN Memory Read Bandwidth") derived from `hns_mc_reqs_local_all`. That name was misleading (the event counts read+write) and, more importantly, the metric was incomplete: it ignored remote (cross-chiplet) memory traffic and surfaced none of the other CMN mesh telemetry the hardware exposes. An earlier attempt to add DRAM read/write bandwidth via the external Arm cmn-tools repo (port-filtered XP watchpoints, cloned at install time) produced unusable numbers on Phoenix — write bandwidth reported as 0 and mesh bandwidth of ~680 GB/s — and added network/`/dev/mem`/kernel-module fragility to the install path. **Why** Root cause of the cmn-tools garbage: Meta fleet telemetry (dynolog) holds ~1.3k CMN perf_event fds continuously, starving the CMN DTC's small hardware counter budget. cmn-tools' watchpoint programming collides with the arm-cmn driver under that contention. Named perf events do not — perf multiplexes and scales them, so steady-state values stay accurate. Separately, the Cypress HN-S MC-request event genuinely has no read/write split (confirmed against the event set and the Phoenix SoC Architecture Specification, Table 15-4), so a DRAM byte-level read/write split is not observable from HN-S counters at all. **Fix** Drop the cmn-tools apparatus entirely (install_platform_tools.sh, cmn_mesh_discovery.sh, cmn_topdown_runner.sh, the install.py / topdown.py hooks, and the log-parsing path in the report generator). Instead report native CMN PMU metrics mapped to the "HNF Effectiveness" group in the Phoenix SoC Architecture Specification, Table 15-4: - **CMN Memory Bandwidth (MBps)** — read+write total, now summing local **and** remote MC requests across both chiplets (each 64B request counted once at its home HN-S, so no double-counting). - **CMN Local / Remote Memory Bandwidth (MBps)** — NUMA locality split. - **CMN Memory Read Mix %** — read share via HN-S QoS PoCQ occupancy (the only read/write proxy the HN-S exposes). - **CMN MC Retry %** — memory-controller backpressure (Table 15-4 mc_retry). - **CMN Snoop Filter Hit Rate %** — coherence-directory effectiveness (sf_hit_ratio). - **CMN Mesh Frequency (GHz)** — from dtc_cycles, for DVFS-throttling visibility. The collect script keeps the CMN event group intentionally lean because of the dynolog counter contention described above. Differential Revision: D102244354
marziehlenjaniMeta
force-pushed
the
export-D102244354-to-v2-beta
branch
from
July 9, 2026 18:01
91818e3 to
785600f
Compare
marziehlenjaniMeta
pushed a commit
to marziehlenjaniMeta/DCPerf
that referenced
this pull request
Jul 9, 2026
…#605) Summary: **Problem** The Neoverse V3 perf report exposed a single CMN memory metric ("CMN Memory Read Bandwidth") derived from `hns_mc_reqs_local_all`. That name was misleading (the event counts read+write) and, more importantly, the metric was incomplete: it ignored remote (cross-chiplet) memory traffic and surfaced none of the other CMN mesh telemetry the hardware exposes. An earlier attempt to add DRAM read/write bandwidth via the external Arm cmn-tools repo (port-filtered XP watchpoints, cloned at install time) produced unusable numbers on Phoenix — write bandwidth reported as 0 and mesh bandwidth of ~680 GB/s — and added network/`/dev/mem`/kernel-module fragility to the install path. **Why** Root cause of the cmn-tools garbage: Meta fleet telemetry (dynolog) holds ~1.3k CMN perf_event fds continuously, starving the CMN DTC's small hardware counter budget. cmn-tools' watchpoint programming collides with the arm-cmn driver under that contention. Named perf events do not — perf multiplexes and scales them, so steady-state values stay accurate. Separately, the Cypress HN-S MC-request event genuinely has no read/write split (confirmed against the event set and the Phoenix SoC Architecture Specification, Table 15-4), so a DRAM byte-level read/write split is not observable from HN-S counters at all. **Fix** Drop the cmn-tools apparatus entirely (install_platform_tools.sh, cmn_mesh_discovery.sh, cmn_topdown_runner.sh, the install.py / topdown.py hooks, and the log-parsing path in the report generator). Instead report native CMN PMU metrics mapped to the "HNF Effectiveness" group in the Phoenix SoC Architecture Specification, Table 15-4: - **CMN Memory Bandwidth (MBps)** — read+write total, now summing local **and** remote MC requests across both chiplets (each 64B request counted once at its home HN-S, so no double-counting). - **CMN Local / Remote Memory Bandwidth (MBps)** — NUMA locality split. - **CMN Memory Read Mix %** — read share via HN-S QoS PoCQ occupancy (the only read/write proxy the HN-S exposes). - **CMN MC Retry %** — memory-controller backpressure (Table 15-4 mc_retry). - **CMN Snoop Filter Hit Rate %** — coherence-directory effectiveness (sf_hit_ratio). - **CMN Mesh Frequency (GHz)** — from dtc_cycles, for DVFS-throttling visibility. The collect script keeps the CMN event group intentionally lean because of the dynolog counter contention described above. Differential Revision: D102244354
marziehlenjaniMeta
force-pushed
the
export-D102244354-to-v2-beta
branch
from
July 9, 2026 18:03
785600f to
82f960f
Compare
meta-codesync
Bot
changed the title
marziehlenjaniMeta
pushed a commit
to marziehlenjaniMeta/DCPerf
that referenced
this pull request
Jul 9, 2026
…#605) Summary: **Problem** The Neoverse V3 perf report exposed a single CMN memory metric ("CMN Memory Read Bandwidth") derived from `hns_mc_reqs_local_all`. That name was misleading (the event counts read+write) and, more importantly, the metric was incomplete: it ignored remote (cross-mesh) memory traffic and surfaced none of the other CMN mesh telemetry the hardware exposes. An earlier attempt to add DRAM read/write bandwidth via the external Arm cmn-tools repo (port-filtered crosspoint watchpoints, cloned at install time) produced unusable numbers on this platform -- write bandwidth reported as 0 and implausibly high mesh bandwidth -- and added network / `/dev/mem` / kernel-module fragility to the install path. **Why** Root cause of the cmn-tools garbage: on a typical deployment, system PMU-monitoring daemons hold many CMN perf_event handles continuously, starving the CMN DTC's small hardware counter budget. cmn-tools' watchpoint programming collides with the arm-cmn driver under that contention. Named perf events do not -- perf multiplexes and scales them, so steady-state values stay accurate. Separately, the HN-S MC-request event has no read/write split (confirmed against the Arm CMN event set), so a DRAM byte-level read/write split is not observable from HN-S counters at all. **Fix** Drop the cmn-tools apparatus entirely (the platform-tools install hook, mesh discovery and topdown-runner scripts, and the log-parsing path in the report generator). Instead report native CMN PMU metrics that correspond to the HN-S "effectiveness" event group in the Arm CMN Technical Reference Manual: - **CMN Memory Bandwidth (MBps)** -- read+write total, now summing local **and** remote MC requests across all mesh instances (each 64B request counted once at its home HN-S, so no double-counting). - **CMN Local / Remote Memory Bandwidth (MBps)** -- memory locality split. - **CMN Memory Read Mix %** -- read share via HN-S QoS PoCQ occupancy (the only read/write proxy the HN-S exposes). - **CMN MC Retry %** -- memory-controller backpressure (mc_retry). - **CMN Snoop Filter Hit Rate %** -- coherence-directory effectiveness (sf_hit_ratio). - **CMN Mesh Frequency (GHz)** -- from dtc_cycles, for DVFS-throttling visibility. The collect script keeps the CMN event group intentionally lean because of the counter contention described above. Differential Revision: D102244354
marziehlenjaniMeta
force-pushed
the
export-D102244354-to-v2-beta
branch
from
July 9, 2026 18:09
82f960f to
abce436
Compare
marziehlenjaniMeta
pushed a commit
to marziehlenjaniMeta/DCPerf
that referenced
this pull request
Jul 9, 2026
…#605) Summary: **Problem** The Neoverse V3 perf report exposed a single CMN memory metric ("CMN Memory Read Bandwidth") derived from `hns_mc_reqs_local_all`. That name was misleading (the event counts read+write) and, more importantly, the metric was incomplete: it ignored remote (cross-mesh) memory traffic and surfaced none of the other CMN mesh telemetry the hardware exposes. An earlier attempt to add DRAM read/write bandwidth via the external Arm cmn-tools repo (port-filtered crosspoint watchpoints, cloned at install time) produced unusable numbers on this platform -- write bandwidth reported as 0 and implausibly high mesh bandwidth -- and added network / `/dev/mem` / kernel-module fragility to the install path. **Why** Root cause of the cmn-tools garbage: on a typical deployment, system PMU-monitoring daemons hold many CMN perf_event handles continuously, starving the CMN DTC's small hardware counter budget. cmn-tools' watchpoint programming collides with the arm-cmn driver under that contention. Named perf events do not -- perf multiplexes and scales them, so steady-state values stay accurate. Separately, the HN-S MC-request event has no read/write split (confirmed against the Arm CMN event set), so a DRAM byte-level read/write split is not observable from HN-S counters at all. **Fix** Drop the cmn-tools apparatus entirely (the platform-tools install hook, mesh discovery and topdown-runner scripts, and the log-parsing path in the report generator). Instead report native CMN PMU metrics that correspond to the HN-S "effectiveness" event group in the Arm CMN Technical Reference Manual: - **CMN Memory Bandwidth (MBps)** -- read+write total, now summing local **and** remote MC requests across all mesh instances (each 64B request counted once at its home HN-S, so no double-counting). - **CMN Local / Remote Memory Bandwidth (MBps)** -- memory locality split. - **CMN Memory Read Mix %** -- read share via HN-S QoS PoCQ occupancy (the only read/write proxy the HN-S exposes). - **CMN MC Retry %** -- memory-controller backpressure (mc_retry). - **CMN Snoop Filter Hit Rate %** -- coherence-directory effectiveness (sf_hit_ratio). - **CMN Mesh Frequency (GHz)** -- from dtc_cycles, for DVFS-throttling visibility. The collect script keeps the CMN event group intentionally lean because of the counter contention described above. Differential Revision: D102244354
marziehlenjaniMeta
force-pushed
the
export-D102244354-to-v2-beta
branch
from
July 9, 2026 19:27
abce436 to
ea0421c
Compare
Differential Revision: D102244354
meta-codesync
Bot
changed the title
marziehlenjaniMeta
force-pushed
the
export-D102244354-to-v2-beta
branch
from
August 8, 2026 18:51
ea0421c to
60f5cd9
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Differential Revision: D102244354