Vol 1 camera-ready: fix accuracy findings from full review pass

Corrects verified factual errors and internal inconsistencies caught in a
per-chapter accuracy review of Volume 1. All changed numbers stay
LEGO-computed; no prose literals introduced.

Critical:
- appendix (System Assumptions): mixed-precision Adam memory table now renders
  master+moments = 12 B and total = 16 B (was 8/12; total had omitted the FP32
  master weight), matching the table caption, prose, and napkin math.

Factual / internal-consistency fixes:
- hw_acceleration: AlexNet ImageNet-2012 margin 10.8 -> 10.9 pp; GPU-supply
  growth renders 1.7x (dropped a stray round() that showed 2x); H100 FP8
  dense/sparse peaks labeled; H100 FP16-vs-FP32 gap ~15x.
- model_compression: NVIDIA T4 mislabeled as an 8 GB card -> RTX 3070; NAS
  GPU-day table clarified vs landmark-system prose; prune+INT8 speedup scope;
  ResNet-50 INT8 accuracy pair standardized.
- conclusion: 1024-GPU cluster MTBF "sub-day" -> "days-scale" (chapter computes
  48.8 h); "microwatt" -> "milliwatt".
- ml_workflow: lifecycle-feedback caption routes to data collection (matching
  the figure arrow and prose); transfer-learning corpus -> ILSVRC-2012 subset.
- data_selection: optimization-triad figure bottom edge now "I/O Bound" (was a
  duplicate "Sample Efficiency").
- data_engineering: Apache Pulsar reclassified CP (was AP); Jeff Dean latency
  numbers -> LADIS 2009 keynote; KWS model-size budget 64 KB.
- benchmarking: MFU peak-vs-sustained gap computed (2-3.3x); WILDS and DataComp
  claims reconciled to one framing.
- model_serving: Hazelwood figure "translation directions" (was "language
  pairs"); batch-8 throughput 4.4x (computed from the saturated model).
- introduction: Viola-Jones 12 features; Perceptron 1958; MYCIN "Bacteroides";
  ViT ImageNet top-5 ~99%.
- nn_computation / nn_architectures: HOG 4x4-vs-8x8 bridging clause; FP16/BF16
  exponent-range clause attached to BF16 only; Perceptron "-tron" etymology;
  TPU v1 "deployed 2015, detailed 2017".
- frameworks: Keras/JAX shape-tuple bug (input_shape=(10,), (1,)).
- training: loss-scaling factor start-vs-steady-state wording.
- ml_systems: energy-gap footnote anchors aligned to chapter values; cost-gap
  code comment 6 orders; Mobile cost cell clarified.
- ml_ops: A100 L2 bandwidth described as approximate, not vendor-published.
- responsible_engr: footnote 43x -> computed 43.1x reference.
- appendix (Data): PSI worked-example addends now sum exactly; softmax-overflow
  wording softened; B200 4-bit row labeled FP4.
- appendix (Machine): DRAM-vs-FP16-multiply summary uses the computed ratio ref.

Registry:
- mlsysim tech memory: register-file access energy 0.01 -> 0.1 pJ. The old
  value was an unsourced outlier (a 64,000x register-to-DRAM energy gap and a
  register 50x cheaper than L1); 0.1 pJ gives the standard ~6,400x DRAM gap and
  register ~5x cheaper than L1. Consumers are computed refs and update
  consistently.

Verified: all Vol 1 LEGO cells exec-pass; binder gates (refs, labels, math,
code, footnotes, notation, tables, structure) green; Vol 1 PDF builds with no
unresolved refs, no Python tracebacks, and no severe layout overflow; the
optimization-triad figure was visually confirmed in the rendered PDF.

Author: profvjreddi

book/quarto/contents/vol1/backmatter/appendix_assumptions.qmd

@@ -345,7 +345,7 @@ These are the **accelerator assumptions**\index{Assumptions!accelerator} used in
 |:-----------------------------------|:---------------------------------------------------------:|:----------------------------------------------------------:|
 | Peak FP16 tensor throughput (B200) |   `{python} AcceleratorConstants.b200_peak_fp16_val_str`  |  `{python} AcceleratorConstants.b200_peak_fp16_unit_str`   |
 | Peak FP8 tensor throughput (B200)  |      `{python} AcceleratorConstants.b200_fp8_val_str`     |     `{python} AcceleratorConstants.b200_fp8_unit_str`      |
-| Peak INT4 throughput (B200)        |   `{python} AcceleratorConstants.b200_int4_tops_val_str`  |  `{python} AcceleratorConstants.b200_int4_tops_unit_str`   |
+| Peak FP4 throughput (B200)         |   `{python} AcceleratorConstants.b200_int4_tops_val_str`  |  `{python} AcceleratorConstants.b200_int4_tops_unit_str`   |
 | HBM bandwidth (B200)               |    `{python} AcceleratorConstants.b200_mem_bw_val_str`    |    `{python} AcceleratorConstants.b200_mem_bw_unit_str`    |
 | HBM capacity (B200)                | `{python} AcceleratorConstants.b200_mem_capacity_val_str` | `{python} AcceleratorConstants.b200_mem_capacity_unit_str` |
 | TDP (B200)                         |      `{python} AcceleratorConstants.b200_tdp_val_str`     |     `{python} AcceleratorConstants.b200_tdp_unit_str`      |
@@ -480,17 +480,17 @@ class MemoryConstants:
     BYTES_FP16_unit_str = fmt_unit(BYTES_FP16)
     BYTES_FP32_val_str = fmt_val(BYTES_FP32)
     BYTES_FP32_unit_str = fmt_unit(BYTES_FP32)
-    BYTES_ADAM_STATE_val_str = fmt_val(BYTES_ADAM_STATE)
-    BYTES_ADAM_STATE_unit_str = fmt_unit(BYTES_ADAM_STATE)
-    ADAM_MIXED_PREC_TOTAL_val_str = fmt_val(BYTES_FP16 + BYTES_FP16 + BYTES_ADAM_STATE)
+    MASTER_PLUS_ADAM_val_str = fmt_val(BYTES_FP32 + BYTES_ADAM_STATE)
+    MASTER_PLUS_ADAM_unit_str = fmt_unit(BYTES_FP32 + BYTES_ADAM_STATE)
+    ADAM_MIXED_PREC_TOTAL_val_str = fmt_val(BYTES_FP16 + BYTES_FP16 + BYTES_FP32 + BYTES_ADAM_STATE)
     ADAM_MIXED_PREC_TOTAL_unit_str = fmt_unit(BYTES_FP16)
 ```
 
 | **Assumption**                             |                        **Value**                         |                          **Unit**                         |
 |:-------------------------------------------|:--------------------------------------------------------:|:---------------------------------------------------------:|
 | Weight/gradient width (BF16)               |      `{python} MemoryConstants.BYTES_FP16_val_str`       |       `{python} MemoryConstants.BYTES_FP16_unit_str`      |
 | Master weight width (FP32)                 |      `{python} MemoryConstants.BYTES_FP32_val_str`       |       `{python} MemoryConstants.BYTES_FP32_unit_str`      |
-| Master + Adam states per parameter (FP32)  |   `{python} MemoryConstants.BYTES_ADAM_STATE_val_str`    |    `{python} MemoryConstants.BYTES_ADAM_STATE_unit_str`   |
+| Master + Adam states per parameter (FP32)  |   `{python} MemoryConstants.MASTER_PLUS_ADAM_val_str`    |    `{python} MemoryConstants.MASTER_PLUS_ADAM_unit_str`   |
 | Bytes per parameter (mixed-precision Adam) | `{python} MemoryConstants.ADAM_MIXED_PREC_TOTAL_val_str` | `{python} MemoryConstants.ADAM_MIXED_PREC_TOTAL_unit_str` |
 
 : **Training Memory Conventions**: Per-parameter storage for mixed-precision Adam (2 + 2 + 12 = 16 bytes before activations). Book convention for napkin math; see [@nvidia_tensors_fp16_2017] for mixed-precision training context. {#tbl-assumptions-training-memory tbl-colwidths="[50,20,30]"}

book/quarto/contents/vol1/backmatter/appendix_data.qmd

@@ -143,7 +143,7 @@ class SerializationCost:
     csv_cycles = 100
     parquet_cycles = 10
     proto_speed = 300 * (MB / second)
-    proto_cycles = 200
+    proto_cycles = 50
 
     # ┌── 2. EXECUTE (The Compute) ─────────────────────────
     speedup = (parquet_speed / csv_speed).to("").magnitude
@@ -481,7 +481,7 @@ class KLDrift:
     kl_pq_t_fmt = [f"{t:.4f}" for t in kl_pq_t]
     kl_qp_t_fmt = [f"{t:.4f}" for t in kl_qp_t]
     diff_fmt = [f"{d:.2f}" for d in diffs]
-    psi_t_fmt = [f"{t:.4f}" for t in psi_t]
+    psi_t_fmt = [f"{t:.5f}" for t in psi_t]
 
     # Scalar prose exports
     kl_pq_str = fmt(kl_pq, precision=4, commas=False)
@@ -754,7 +754,7 @@ $$
 \text{`{python} LogSumExp.naive_exp0_eq`},\quad \text{`{python} LogSumExp.naive_exp1_eq`}, \quad \text{`{python} LogSumExp.naive_exp2_eq`}
 $$
 
-These numbers are representable in FP64 but overflow FP32 (max $\approx 3.4 \times 10^{38}$). With FP16 (max $\approx 65{,}504$), even $e^{12}$ overflows. In practice, logits of magnitude 100 are not unusual in deep networks, and FP16/BF16 is the standard training precision—so naive softmax fails routinely.
+These numbers are representable in FP64 but overflow FP32 (max $\approx 3.4 \times 10^{38}$). With FP16 (max $\approx 65{,}504$), even $e^{12}$ overflows. Raw logits rarely reach magnitude 100, but large intermediate values do arise (for example, unscaled attention scores or overconfident outputs), and because FP16 and BF16 are the standard training precisions, a naive softmax risks overflow often enough that the stable form is used by default.
 
 **With the trick**: Subtract $a = \max(z) =$ `{python} LogSumExp.a_str`:
 \begin{gather*}

book/quarto/contents/vol1/backmatter/appendix_machine.qmd

@@ -1237,7 +1237,7 @@ With the full toolkit assembled---reference numbers, performance models, archite
 
 ::: {.callout-takeaways title="Numbers every engineer should know"}
 
-* **Energy dominates**: In the cited 45 nm reference, a 32-bit DRAM read uses roughly 580$\times$ the energy of one FP16 multiply. The ratio varies by platform, but it makes arithmetic intensity and reuse central design metrics.
+* **Energy dominates**: In the cited 45 nm reference, a 32-bit DRAM read uses roughly `{python} NumbersToKnow.dram_vs_compute_ratio_mult_str` the energy of one FP16 multiply. The ratio varies by platform, but it makes arithmetic intensity and reuse central design metrics.
 * **The Roofline model**: The Roofline Model shows whether compute throughput or memory bandwidth is the active ceiling. Batch-1 weight-streaming inference is often below the ridge; batching can raise intensity, but shape, precision, reuse, and the ridge point determine whether it crosses.
 * **Amdahl's Law**: Amdahl's Law caps strong-scaling speedup at $1/s$ (where $s$ is the serial fraction). Gustafson's Law shows that scaling the problem alongside hardware yields near-linear throughput gains—the paradigm that makes large-scale training feasible.
 * **Little's Law**: $Q_{\text{req}} = \lambda_{\text{arr}} T_{\text{lat}}$ links mean population, throughput, and time in system; memory also depends on where queued and executing state resides.

book/quarto/contents/vol1/benchmarking/benchmarking.qmd

@@ -68,7 +68,7 @@ from mlsysim.core.units import *
 from mlsysim import Hardware, Engine, Models
 from mlsysim.solvers import EfficiencyModel
 from mlsysim.core.units import TFLOP, second
-from mlsysim.fmt import fmt_flop_rate, fmt_percent, fmt_time, fmt_multiple, check
+from mlsysim.fmt import fmt_flop_rate, fmt_percent, fmt_time, fmt_multiple, fmt_multiple_range, check
 
 class A100PeakBenchmark:
     # ┌── 1. LOAD (Constants) ───────────────────────────────────────────────
@@ -84,6 +84,10 @@ class A100PeakBenchmark:
     # Calculate optimally tuned MFU (e.g. 50% MFU upper bound from literature)
     eff_tuned = EfficiencyModel().solve(m, h, workload_type="ffn", use_flash_attention=False, precision="fp16", efficiency=0.5)
 
+    # Sustained-to-peak gap = peak / sustained = 1 / MFU
+    gap_low = 1 / eff_tuned.mfu  # 50% MFU ceiling -> 2.0x
+    gap_high = 1 / eff_base.mfu  # 30% MFU floor -> 3.3x
+
     # ┌── 3. GUARD (Invariants) ───────────────────────────────────────────
     check(eff_base.mfu == 0.3, "Base attention should be 30% MFU.")
     check(eff_tuned.mfu == 0.5, "FFN should be 50% MFU.")
@@ -94,13 +98,14 @@ class A100PeakBenchmark:
     mfu_tuned_pct_str = fmt_percent(eff_tuned.mfu, precision=0, commas=False, style="prose")
     achievable_base_tflop_str = fmt_flop_rate(eff_base.achievable_flops, unit=TFLOP / second, precision=1, commas=False)
     achievable_tuned_tflop_str = fmt_flop_rate(eff_tuned.achievable_flops, unit=TFLOP / second, precision=0, commas=False)
+    sustained_gap_range_mult_str = fmt_multiple_range(gap_low, gap_high, precision=(0, 1))
 ```
 
 ::: {#dfn-benchmarking-machine-learning-benchmarking .callout-definition title="Machine learning benchmarking"}
 
 ***Machine Learning Benchmarking***\index{Benchmarking!definition} is the empirical measurement of a system's end-to-end performance on representative ML workloads, designed to decouple marketed peak specifications from the sustained throughput and latency achievable under realistic operating conditions.
 
-1.  **Significance**: The gap between peak and sustained performance is large and structurally unavoidable. An A100 GPU delivers `{python} A100PeakBenchmark.a100_fp16_tflop_per_s_str` (BF16) at peak, but production transformer training runs typically sustain `{python} A100PeakBenchmark.achievable_base_tflop_str`--`{python} A100PeakBenchmark.achievable_tuned_tflop_str` (`{python} A100PeakBenchmark.mfu_base_pct_str`--`{python} A100PeakBenchmark.mfu_tuned_pct_str` MFU), about a 2--3.5$\times$ gap that exists even in optimally tuned systems due to memory stalls, pipeline bubbles, and kernel launch overhead. Benchmarking quantifies this $\eta_{\text{hw}}$ gap; vendor spec sheets do not.
+1.  **Significance**: The gap between peak and sustained performance is large and structurally unavoidable. An A100 GPU delivers `{python} A100PeakBenchmark.a100_fp16_tflop_per_s_str` (BF16) at peak, but production transformer training runs typically sustain `{python} A100PeakBenchmark.achievable_base_tflop_str`--`{python} A100PeakBenchmark.achievable_tuned_tflop_str` (`{python} A100PeakBenchmark.mfu_base_pct_str`--`{python} A100PeakBenchmark.mfu_tuned_pct_str` MFU), about a `{python} A100PeakBenchmark.sustained_gap_range_mult_str` gap that exists even in optimally tuned systems due to memory stalls, pipeline bubbles, and kernel launch overhead. Benchmarking quantifies this $\eta_{\text{hw}}$ gap; vendor spec sheets do not.
 2.  **Distinction**: Unlike micro-benchmarks, which measure individual kernel performance such as a general matrix multiply (GEMM) at peak matrix dimensions, ML benchmarks measure the full stack: data loading, preprocessing, forward pass, gradient computation, optimizer step, and checkpoint I/O—exposing bottlenecks that individual-component benchmarks will never reveal.
 3.  **Common pitfall**: A frequent misconception is that benchmark numbers are stable references. Both the workload (new model architectures) and the hardware (new GPU generations) evolve, so a result that leads a benchmark under one version often becomes the baseline under a later version, making year-over-year comparisons meaningful only when the benchmark version is held constant.
 
@@ -4127,7 +4132,7 @@ The final category of data benchmarking asks whether models will generalize from
 
 The standard assumption underlying held-out evaluation, that test data comes from the same distribution as training data, is routinely violated in practice. Test sets constructed years after training data may reflect distribution drift as the world changes. Test sets from different geographic regions may reflect population shift. A model with strong held-out accuracy can drop sharply when deployed to a region or time period the test set did not represent. Standard held-out evaluation overestimates deployment performance whenever the i.i.d. (independent and identically distributed) assumption fails.
 
-The true test is train-to-production alignment, and this is far harder to measure because production data differs from training data in ways that held-out test sets often fail to capture. Production images come from different cameras with different characteristics. Production users come from different populations with different behaviors. Production inputs include edge cases that curated test sets systematically exclude. The WILDS[^fn-wilds] benchmark [@koh2021wilds] was designed specifically to evaluate models under realistic distribution shifts: hospital systems with different patient populations, wildlife cameras at different locations, satellite imagery from different time periods. The results reveal a stark reality: models achieving 90 percent+ accuracy on in-distribution test sets may drop to 60 percent under these realistic shifts.
+The true test is train-to-production alignment, and this is far harder to measure because production data differs from training data in ways that held-out test sets often fail to capture. Production images come from different cameras with different characteristics. Production users come from different populations with different behaviors. Production inputs include edge cases that curated test sets systematically exclude. The WILDS[^fn-wilds] benchmark [@koh2021wilds] was designed specifically to evaluate models under realistic distribution shifts: hospital systems with different patient populations, wildlife cameras at different locations, satellite imagery from different time periods. The results reveal a stark reality: models achieving 97 percent accuracy on in-distribution test sets may drop to 70 percent under these realistic shifts.
 
 [^fn-wilds]: **WILDS**: Stanford's 2021 benchmark of ten datasets with real-world distribution shifts: hospital patient population changes (Camelyon17), wildlife camera location shifts (iWildCam), and satellite imagery temporal drift (PovertyMap). WILDS quantifies the deployment gap: models achieving 97 percent in-distribution accuracy can drop to 70 percent under these realistic shifts, demonstrating that standard held-out evaluation systematically overestimates production performance when the i.i.d. assumption\index{I.I.D. Assumption!held-out evaluation limitation} fails. \index{WILDS!distribution shift benchmark}
 
@@ -4287,7 +4292,7 @@ anchor=north,fill=BackColor]{\textbf{Model-centric AI}};
 
 Data-centric AI\index{Data-Centric AI!dataset enhancement paradigm} reflects an important shift in understanding that challenges the "more data is always better" assumption: *better* datasets, not just *larger* ones, produce more reliable and generalizable AI systems. Initiatives like DataPerf\index{DataPerf!data quality benchmark} [@mazumder2024dataperf] and DataComp\index{DataComp!data-centric benchmark}[^fn-datacomp] have emerged to systematically evaluate how dataset improvements affect model performance. For instance, DataComp [@beaumont2023] demonstrated that models trained on a carefully curated 30 percent subset of data achieved better results than those trained on the complete dataset, challenging the assumption that more data automatically leads to better performance.
 
-[^fn-datacomp]: **DataComp**: Introduced in 2023, DataComp inverts the standard benchmark by fixing the model and training code, letting participants compete on dataset curation alone. Results showed that a carefully filtered 30 percent subset matched models trained on 10$\times$ larger unfiltered data, quantifying a systems insight: for many workloads, engineering the data pipeline yields greater performance gains per dollar than scaling compute. \index{DataComp!data-centric benchmarking}
+[^fn-datacomp]: **DataComp**: Introduced in 2023, DataComp inverts the standard benchmark by fixing the model and training code, letting participants compete on dataset curation alone. Results showed that a carefully filtered 30 percent subset outperformed models trained on the full unfiltered pool, quantifying a systems insight: for many workloads, engineering the data pipeline yields greater performance gains per dollar than scaling compute. \index{DataComp!data-centric benchmarking}
 
 A persistent challenge in data benchmarking emerges from dataset saturation\index{Dataset Saturation!performance capability illusion}. When models achieve near-perfect accuracy on benchmarks like ImageNet, practitioners must distinguish whether performance gains represent genuine capability advances or merely optimization to existing test sets. As the timeline in @fig-dataset-saturation illustrates, widely tracked AI benchmarks have repeatedly crossed reported human baselines, making each corresponding benchmark less useful as a differentiator [@perrault2024artificial].
 

book/quarto/contents/vol1/conclusion/conclusion.qmd

@@ -104,7 +104,7 @@ The five Lighthouse workloads expose distinct constraint regimes:
 - **GPT-2/Llama**\index{GPT-2!memory bandwidth}: Autoregressive language generation exposes the opposite wall, where every token reloads enough state that memory bandwidth, KV-cache growth, and model parallelism dominate serving cost.
 - **MobileNetV2**\index{MobileNetV2!efficiency under constraint}: Depthwise separable convolutions and INT8 quantization trade representational capacity for mobile NPU deployment in a power-constrained regime.
 - **DLRM**\index{DLRM!capacity-bound workload}: Terabyte-scale embedding tables shift the binding constraint from memory *bandwidth* to memory *capacity*, forcing engineers to design around where data physically resides and how sparse operations behave.
-- **Keyword spotting (KWS)/Wake Vision**\index{Keyword Spotting!sub-megabyte models}: Sub-megabyte models running on microcontrollers with always-on inference under microwatt power budgets make every byte and every milliwatt matter.
+- **Keyword spotting (KWS)/Wake Vision**\index{Keyword Spotting!sub-megabyte models}: Sub-megabyte models running on microcontrollers with always-on inference under milliwatt power budgets make every byte and every milliwatt matter.
 
 Together, these five workloads span the full deployment spectrum from data center to microcontroller, probing the bottlenecks these principles diagnose and testing every optimization strategy the book has taught. The systems thinking we developed by tracing these Lighthouses across chapters, from architecture design through training, optimization, and deployment, is the integrated perspective that distinguishes ML systems engineering from isolated algorithm development.
 
@@ -1058,7 +1058,7 @@ The next frontier is scale: models no longer fit on one machine, failures become
 * **Assumptions matter across implementations**: The thirteen principles turn framework-specific craft into measurable reasoning by combining physical bounds, decompositions, fitted models, requirements, and heuristics. Apply each only within its stated scope.
 * **Trace displaced costs without assuming conservation**: Compression, batching, monitoring, and governance can relocate burdens across data, algorithm, and machine, while good design can remove accidental complexity outright. Measurement distinguishes the two cases.
 * **Boundaries reveal the bottleneck**: A 70-billion-parameter Llama 2 can be about `{python} ConclusionRoofline.ratio_mult_str` memory-bound on H100, and p99 latency can sit `{python} TailLatencyRatio.conclusion_tail_ratio_mult_str` above the mean. Systems thinking means measuring where physics, traffic, and users bind.
-* **Scale changes the binding term**: The next frontier is scale, where a thousand-GPU pool turns multi-year component MTTF into sub-day cluster MTBF. The physics stays, but the constraint moves to fleets.
+* **Scale changes the binding term**: The next frontier is scale, where a thousand-GPU pool turns multi-year component MTTF into days-scale cluster MTBF. The physics stays, but the constraint moves to fleets.
 
 :::