Restore inline bold heading tags

Author: profvjreddi

mlsysim/paper/paper.tex

@@ -802,7 +802,7 @@ \subsection{Algorithm (Scaling \& Compression)}
 r_{\text{quant}} &= \frac{b_{\text{base}}}{b_{\text{target}}} \quad \text{(quantization ratio, e.g., } 16/4 = 4{\times}\text{)} \\
 r_{\text{prune}} &= \frac{1}{1 - s} \quad \text{(memory reduction at sparsity } s\text{)}
 \end{align}
-Here $b_{\text{base}}$ is the baseline precision (typically 16 for FP16/BF16 models, not 32) and $b_{\text{target}}$ is the quantized precision. Quantization reduces memory reads but not FLOPs, shifting the arithmetic intensity rightward on the Roofline by a factor of $r_{\text{quant}}$ and potentially crossing the ridge point from memory-bound to compute-bound. For pruning, $r_{\text{prune}}$ gives the \emph{memory reduction} ratio; actual compute speedup depends on sparsity structure. Unstructured sparsity yields no acceleration on current GPUs, while 2:4 structured sparsity~\citep{nvidia2023h100} provides a $2{\times}$ throughput gain via Sparse Tensor Cores. Post-training quantization methods such as GPTQ~\citep{frantar2023gptq} and AWQ~\citep{lin2025} demonstrate that 4-bit quantization can preserve most accuracy for large language models, making $r_{\text{quant}} = 4{\times}$ a practical operating point. However, compression ratio alone does not determine inference speedup. Unstructured pruning reduces storage but yields no compute speedup on GPU GEMM kernels; only structured pruning and $N{:}M$ sparsity (e.g., 2:4 on NVIDIA Ampere+) accelerate hardware execution. The \texttt{CompressionModel} reports both \texttt{memory\_savings\_pct} and \texttt{inference\_speedup} to capture this distinction. The accuracy impact $\Delta_{\text{acc}}$ is modeled as a configurable function, since the fidelity--compression frontier varies by architecture and task. We assume accuracy degradation follows empirical curves from~\citet{gholami2022}, and pruning compute speedup requires structured sparsity with hardware support.
+Here $b_{\text{base}}$ is the baseline precision (typically 16 for FP16/BF16 models, not 32) and $b_{\text{target}}$ is the quantized precision. Quantization reduces memory reads but not FLOPs, shifting the arithmetic intensity rightward on the Roofline by a factor of $r_{\text{quant}}$ and potentially crossing the ridge point from memory-bound to compute-bound. For pruning, $r_{\text{prune}}$ gives the \emph{memory reduction} ratio; actual compute speedup depends on sparsity structure. Unstructured sparsity yields no acceleration on current GPUs, while 2:4 structured sparsity~\citep{nvidia2023h100} provides a $2{\times}$ throughput gain via Sparse Tensor Cores. Post-training quantization methods such as GPTQ~\citep{frantar2023gptq} and AWQ~\citep{lin2025} demonstrate that 4-bit quantization can preserve most accuracy for large language models, making $r_{\text{quant}} = 4{\times}$ a practical operating point. However, compression ratio alone does not determine inference speedup. Unstructured pruning reduces storage but yields no compute speedup on GPU GEMM kernels; only structured pruning and $N{:}M$ sparsity (e.g., 2:4 on NVIDIA Ampere+) accelerate hardware execution. The \texttt{CompressionModel} reports both \texttt{memory\_savings\_pct} and \texttt{inference\_speedup} to capture this distinction. The accuracy impact $\Delta_{\text{acc}}$ is modeled as a configurable function, since the fidelity--compression frontier varies by architecture and task. \textbf{Assumptions:} Accuracy degradation follows empirical curves from~\citet{gholami2022}; pruning compute speedup requires structured sparsity with hardware support.
 
 \subsection{Fleet (Multi-Node Coordination)}
 \label{sec:walls-fleet}
@@ -881,7 +881,7 @@ \subsection{Operations (Cost, Carbon \& Safety)}
 \text{CO}_2 &= E_{\text{total}} \times \text{CI}_{\text{region}} \quad \text{(gCO}_2\text{/kWh)} \\
 \text{H}_2\text{O} &= E_{\text{total}} \times \text{WUE} \quad \text{(L/kWh)}
 \end{align}
-where PUE is the power usage effectiveness of the datacenter, $\text{CI}_{\text{region}}$ is the carbon intensity of the local grid, and WUE is the water usage effectiveness. This model captures \emph{operational} carbon only. For edge and IoT deployments at scale ($>10^6$ devices), \emph{embodied} carbon from manufacturing can dominate operational carbon by 10--100$\times$~\citep{gupta2022chasing}. The \texttt{SustainabilityModel} accepts an optional \texttt{embodied\_carbon\_per\_device} parameter for lifecycle analysis. We assume grid carbon intensity is a static regional constant; temporal variation (e.g., renewable intermittency) is not modeled. Energy-proportional power follows~\citet{barroso2007}: idle power is 30\% of TDP, with the remaining 70\% scaling linearly with MFU.
+where PUE is the power usage effectiveness of the datacenter, $\text{CI}_{\text{region}}$ is the carbon intensity of the local grid, and WUE is the water usage effectiveness. This model captures \emph{operational} carbon only. For edge and IoT deployments at scale ($>10^6$ devices), \emph{embodied} carbon from manufacturing can dominate operational carbon by 10--100$\times$~\citep{gupta2022chasing}. The \texttt{SustainabilityModel} accepts an optional \texttt{embodied\_carbon\_per\_device} parameter for lifecycle analysis. \textbf{Assumption:} Grid carbon intensity is a static regional constant; temporal variation (e.g., renewable intermittency) is not modeled. Energy-proportional power follows~\citet{barroso2007}: idle power is 30\% of TDP, with the remaining 70\% scaling linearly with MFU.
 
 \textbf{Wall~19: The Checkpoint Wall.} Long-running training jobs must periodically save model state (weights and optimizer states) to persistent storage, incurring an I/O penalty that directly reduces effective MFU. This is a classic manifestation of Amdahl's Law: as fleet size scales to accelerate the parallel forward/backward passes, the serial time spent writing to persistent storage begins to dominate. The \texttt{CheckpointModel} models the I/O burst penalty~\citep{eisenman2022checknrun}:
 \begin{equation}
@@ -1071,7 +1071,7 @@ \subsection{Design-Space Exploration Speed}
 \subsection{Accuracy Scope and Limitations}
 \label{sec:accuracy}
 
-\mlsysim provides first-order estimates, not cycle-accurate predictions. We do not re-derive the resolver abstraction here. It is the same first-principles structural bounds plus a single efficiency~$\eta$ for unmodeled second-order effects, as in \Cref{sec:intro,sec:philosophy}. The efficiency parameter $\eta$ must be selected for the workload class and measurement scope: $\eta \approx 0.19$ for end-to-end ResNet-50 training throughput on A100, reflecting convolution, framework, and data-pipeline overhead~\citep{mattson2020}; $\eta \approx 0.42$ for large-scale Transformer training, consistent with Meta's reported system-level efficiency~\citep{llama3team2024}. The Roofline parallel (peak vs.\ achievable ceilings; which constraint binds) is already stated in \Cref{sec:intro}; see \citet{williams2009}. Two of the seven anchors above require no efficiency parameter at all, because the Chinchilla scaling law is pure mathematics and the carbon footprint is a direct measurement, demonstrating that the physics layer is independently sound. \Cref{sec:discussion} details the specific phenomena this abstraction cannot capture and the resulting accuracy boundaries.
+\mlsysim provides first-order estimates, not cycle-accurate predictions. We do not re-derive the resolver abstraction here. It is the same first-principles structural bounds plus a single efficiency~$\eta$ for unmodeled second-order effects, as in \Cref{sec:intro,sec:philosophy}. \textbf{Calibrating $\eta$ for validation.} The efficiency parameter $\eta$ must be selected for the workload class and measurement scope: $\eta \approx 0.19$ for end-to-end ResNet-50 training throughput on A100, reflecting convolution, framework, and data-pipeline overhead~\citep{mattson2020}; $\eta \approx 0.42$ for large-scale Transformer training, consistent with Meta's reported system-level efficiency~\citep{llama3team2024}. The Roofline parallel (peak vs.\ achievable ceilings; which constraint binds) is already stated in \Cref{sec:intro}; see \citet{williams2009}. Two of the seven anchors above require no efficiency parameter at all, because the Chinchilla scaling law is pure mathematics and the carbon footprint is a direct measurement, demonstrating that the physics layer is independently sound. \Cref{sec:discussion} details the specific phenomena this abstraction cannot capture and the resulting accuracy boundaries.
 
 For \mlsysim's intended use cases (architectural reasoning, lab exercises, capacity planning, and design-space exploration), first-order accuracy is sufficient and often preferable. A student who understands \emph{why} a system is memory-bound has learned more than one who can predict its throughput to three decimal places.