Skip to content

Commit 8fcd0f3

Browse files
authored
Refined the PDF layout for Appendix A (appendix_dam.qmd) (#1830)
1 parent 5530367 commit 8fcd0f3

1 file changed

Lines changed: 5 additions & 5 deletions

File tree

book/quarto/contents/vol1/backmatter/appendix_dam.qmd

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -69,7 +69,7 @@ In the D·A·M acronym, Data, Algorithm, and Machine are taxonomy axes. They are
6969
| **Machine** (pure) | Physics | Silicon, memory hierarchy, peak FLOP/s | @sec-hardware-acceleration |
7070
| $\mathsf{Data} \cap \mathsf{Algorithm} \cap \mathsf{Machine}$ | ML Systems Engineering | iron law, Roofline, training loops, serving | @sec-model-training, @sec-model-serving, @sec-benchmarking |
7171

72-
: **D·A·M Intersection Reference**: Each zone maps specific techniques to the axes they span and the chapters that cover them. The pairwise intersections require reasoning about two domains simultaneously; the center requires all three. {#tbl-dam-intersections tbl-colwidths="[12,25,36,27]"}
72+
: **D·A·M Intersection Reference**: Each zone maps specific techniques to the axes they span and the chapters that cover them. The pairwise intersections require reasoning about two domains simultaneously; the center requires all three. {#tbl-dam-intersections tbl-colwidths="[20,23,30,27]"}
7373

7474
The pure zones contain concepts that belong entirely to one axis: storage formats and distribution properties are purely Data concerns, loss functions and gradient computations are purely Algorithm, and silicon physics and peak FLOP/s are purely Machine. These are the topics where single-domain expertise suffices.
7575

@@ -108,7 +108,7 @@ The systems engineer's job is to make these components run in parallel, not in s
108108
| **Async Gradient Sync** | M (communication) overlaps A | Overlap bucketed AllReduce with remaining backward computation |
109109
| **Double Buffering** | D overlaps M | Fill buffer N+1 while computing on buffer N |
110110

111-
: **D·A·M Overlap Techniques**: Each technique allows one D·A·M axis to execute while another is in flight, converting the iron law's additive terms into overlapped terms. The payoff is transforming $T = a + b$ into $T = \max(a, b)$, which can cut latency nearly in half when the terms are balanced. {#tbl-dam-overlap}
111+
: **D·A·M Overlap Techniques**: Each technique allows one D·A·M axis to execute while another is in flight, converting the iron law's additive terms into overlapped terms. The payoff is transforming $T = a + b$ into $T = \max(a, b)$, which can cut latency nearly in half when the terms are balanced. {#tbl-dam-overlap tbl-colwidths="[22,32,46]"}
112112

113113
Overlap only helps when the D·A·M axes are reasonably balanced. If one term dominates (for example, severely memory bound), overlapping the smaller term with the larger yields negligible gain—the max is still dominated by the same bottleneck. Overlap provides the greatest benefit when $D_{\text{vol}}/\text{BW} \approx O/(R_{\text{peak}} \cdot \eta_{\text{hw}})$. The latency term is the important exception.
114114

@@ -147,12 +147,12 @@ Common industry labels map to D·A·M components as follows: memory bound typica
147147
Once the bottleneck is identified, @tbl-bottleneck-actions shows which optimizations help and which ones are wasted:
148148

149149
| **If the workload is...** | **Dominant Term** | **Optimization That Works** | **Optimization That is Wasted** |
150-
|:--------------------------|:---------------------------------------------|:------------------------------------------------------------------|:---------------------------------------------------|
150+
|:--------------------------|:--------------------------------------------:|:------------------------------------------------------------------|:---------------------------------------------------|
151151
| **Memory-Bound** | $D_{\text{vol}}/\text{BW}$ | Quantization, pruning, batching, kernel fusion | Faster accelerator (more FLOP/s will not help) |
152152
| **Compute-Bound** | $O/(R_{\text{peak}} \cdot \eta_{\text{hw}})$ | Better kernels, Tensor Cores, faster accelerator, lower precision | More memory bandwidth (not the binding term) |
153153
| **Latency-Bound** | $L_{\text{lat}}$ | Batching requests, kernel fusion, async dispatch | Neither compute nor bandwidth (overhead dominates) |
154154

155-
: **What Works vs. What Is Wasted**: Optimizing the wrong term yields exactly zero improvement. A memory-bound large language model (LLM) will not speed up from a faster accelerator; the accelerator will simply idle faster while waiting for memory. {#tbl-bottleneck-actions tbl-colwidths="[17,17,33,33]"}
155+
: **What Works vs. What Is Wasted**: Optimizing the wrong term yields exactly zero improvement. A memory-bound large language model (LLM) will not speed up from a faster accelerator; the accelerator will simply idle faster while waiting for memory. {#tbl-bottleneck-actions tbl-colwidths="[19,17,32,32]"}
156156

157157
Knowing what works also means recognizing what does not. In practice, teams under deadline pressure repeatedly fall into the same traps—optimizing the wrong axis with confidence. These failure modes are common enough to deserve their own names.
158158

@@ -269,7 +269,7 @@ To move beyond qualitative guessing, the efficiency ratios in @tbl-dam-scorecard
269269
[^fn-mfu-scorecard]: **MFU (Model FLOPs Utilization)**: The ratio of achieved model FLOP/s to the hardware's theoretical peak FLOP/s, introduced in the PaLM paper [@chowdhery2022palm]. Unlike raw accelerator utilization (which counts any work the accelerator performs), MFU measures only *useful* model computation rate, excluding overhead like gradient synchronization and memory management. @sec-benchmarking covers MFU in depth.
270270

271271
| **Axis** | **Metric** | **Definition** | **Failing Grade** | **Passing Grade** |
272-
|:--------------|:------------------|:----------------------------------------------------------|:-------------------:|:-----------------------:|
272+
|:--------------|:------------------|:---------------------------------------------------------:|:-------------------:|:-----------------------:|
273273
| **Data** | **I/O Overhead** | $\frac{\text{Data Wait Time}}{\text{Total Step Time}}$ | $>$ 10 percent | $<$ 1 percent |
274274
| **Algorithm** | **Active Params** | $\frac{\text{Nonzero Params}}{\text{Total Params}}$ | 100 percent (Dense) | $<$ 50 percent (Sparse) |
275275
| **Machine** | **MFU** | $\frac{\text{Achieved model FLOP/s}}{\text{Peak FLOP/s}}$ | $<$ 30 percent | $>$ 50 percent |

0 commit comments

Comments
 (0)