Polish Volume I page flow

Author: profvjreddi

book/quarto/contents/vol1/data_selection/data_selection.qmd

@@ -2729,7 +2729,7 @@ With the bottleneck identified, verify that the corresponding techniques are fea
 
 #### Step 3: Estimate ROI {.unnumbered}
 
-Meeting the prerequisites is necessary but not sufficient. Before committing engineering resources, estimate the return on investment for each candidate technique:
+Before committing engineering resources, estimate each candidate technique's return on investment:
 $$
 \text{ROI} = \frac{\text{(Baseline Cost)} - \text{(Technique Cost + Implementation Cost)}}{\text{Technique Cost + Implementation Cost}}
 $$

book/quarto/contents/vol1/frameworks/frameworks.qmd

@@ -4726,7 +4726,7 @@ Engineers assume compilation overhead is a one-time cost that pays off quickly.
 
 **Pitfall**: *Using one execution policy for exploration and production.*
 
-The right framework mode depends on the loop being optimized. During exploration, eager execution and smaller tests often shorten the human feedback loop because they avoid repeated graph captures and recompilations. During production serving or long training runs, compilation can amortize its setup cost across enough requests or samples to pay back. Teams that use the same execution policy in both phases either slow research iteration or leave production throughput on the table.
+The right framework mode depends on the loop. During exploration, eager execution and small tests shorten feedback by avoiding repeated graph captures and recompilations. During production serving or long training runs, compilation can amortize setup across many requests or samples. The deciding factors are workload duration and stability: compilation pays when graph structure and shapes recur, whereas changing control flow or input shapes can invalidate cached artifacts and restart the amortization. Teams should choose the policy from measured iteration latency and steady-state throughput. One policy for both phases either slows research iteration or leaves production throughput unused.
 
 :::
 

book/quarto/contents/vol1/hw_acceleration/hw_acceleration.qmd

@@ -4436,7 +4436,9 @@ Good placement can substantially reduce latency, while poor placement leaves pro
 
 While computation placement determines where operations execute, memory allocation defines where data resides and how it flows through the hierarchy. The goal is to keep reused data near the processing elements without exceeding faster-tier capacity. GPUs expose global memory, shared memory, caches, and registers that kernels coordinate through tiling [@NVIDIA2020]. TPUs use on-chip buffers to stage activations and weights for systolic-array execution (@fig-systolic-array) [@jouppi2017], while wafer-scale processors partition memory and computation to control interconnect traffic [@Cerebras2021]. General-purpose processors and accelerators both use memory hierarchies, but many accelerators expose more placement decisions to compilers and kernels. Poor allocation can therefore impose three related penalties: additional latency, greater energy from higher-tier accesses, and lower throughput when processing elements wait for data.
 
-The severity of these penalties varies by workload. CNNs rely on structured, localized access patterns and benefit from well-defined memory layouts that facilitate predictable reuse [@chen2016eyeriss]. Transformer models require access to large parameter sets and intermediate activations, making them sensitive to capacity and bandwidth. GNNs add irregular sparse structures that complicate allocation and prefetching. @Tbl-memory-allocation summarizes these challenges. Systems combine static memory plans for known shapes with dynamic buffer management when shapes vary, while device capacity limits which models fit without partitioning or offload.
+The severity of these penalties varies by workload. CNNs rely on structured, localized access patterns and benefit from well-defined memory layouts that facilitate predictable reuse [@chen2016eyeriss]. Transformer models require access to large parameter sets and intermediate activations, making them sensitive to capacity and bandwidth. GNNs add irregular sparse structures that complicate allocation and prefetching.
+
+Capacity is only the first allocation test. A tensor can fit in accelerator memory yet still dominate execution if each kernel rereads it from a distant tier; conversely, tiling can keep a larger working set efficient by preserving reuse locally. The allocation diagnostic is therefore traffic, not fit alone: designers must count how often values cross each memory boundary and keep frequently reused values in the fastest feasible tier. @Tbl-memory-allocation summarizes these challenges. Systems combine static memory plans for known shapes with dynamic buffer management when shapes vary, while device capacity limits which models fit without partitioning or offload.
 
 | Challenge                                                                               | Impact on Execution                                                                    | **Key Considerations for Allocation**                                                             |
 |:----------------------------------------------------------------------------------------|:---------------------------------------------------------------------------------------|:--------------------------------------------------------------------------------------------------|

book/quarto/contents/vol1/ml_ops/ml_ops.qmd

@@ -2123,9 +2123,7 @@ For an illustrative GPU at `{python} CostPerInference.hourly_gpu_cost_str` proce
 
 #### Model and infrastructure monitoring {#sec-ml-operations-model-infrastructure-monitoring-3988}
 
-Infrastructure management provisions resources; monitoring\index{Model monitoring!performance tracking}\index{Model monitoring!drift detection} observes their behavior. Unit tests can verify deterministic components but cannot by themselves establish population-level predictive performance, which must be estimated statistically. Monitoring implements observable degradation (@sec-ml-operations-foundational-principles-44e6), transforming this theoretical limitation into operational practice. Once monitoring surfaces a symptom—a latency SLA miss, throughput below target, or memory creep—@sec-appdx-dam-taxonomy-bottleneck-diagnostic-3fc9 maps that symptom to its dominant D·A·M term and tells the operator which optimizations will move the binding constraint and which will be wasted on serving infrastructure. Without continuous monitoring and the deeper **observability**\index{Observability!system state inference}[^fn-observability-ml-metrics] it enables (the ability to infer internal state from outputs), a deployed model is a black box slowly drifting toward irrelevance.
-
-[^fn-observability-ml-metrics]: **Observability**\index{Observability!control theory origin} (from control theory [@kalman1960]): How well internal states can be inferred from outputs. In MLOps, monitoring shows whether a system is degrading; observability helps diagnose why through signals such as feature distributions and prediction confidence.
+Infrastructure management provisions resources; monitoring\index{Model monitoring!performance tracking}\index{Model monitoring!drift detection} observes their behavior. Unit tests can verify deterministic components but cannot by themselves establish population-level predictive performance, which must be estimated statistically. Monitoring implements observable degradation (@sec-ml-operations-foundational-principles-44e6), transforming this theoretical limitation into operational practice. Once monitoring surfaces a symptom—a latency SLA miss, throughput below target, or memory creep—@sec-appdx-dam-taxonomy-bottleneck-diagnostic-3fc9 maps that symptom to its dominant D·A·M term and tells the operator which optimizations will move the binding constraint and which will be wasted on serving infrastructure. Without continuous monitoring and the deeper **observability**\index{Observability!system state inference} it enables (the ability to infer internal state from outputs), a deployed model is a black box slowly drifting toward irrelevance.
 
 Effective monitoring spans both model behavior and infrastructure performance. On the model side, teams track metrics such as accuracy, precision, recall, and the confusion matrix [@scikit_learn_confusion_matrix] using live or sampled predictions to detect whether performance remains stable or begins to drift. A critical constraint is the **drift detection delay**\index{Drift detection delay!statistical monitoring}, which determines how quickly statistical monitoring can confirm that degradation has occurred. The speed of detection depends on traffic volume. A short sample-rate calculation makes that constraint visible.
 
@@ -4160,7 +4158,7 @@ The infrastructure components examined throughout the chapter directly implement
 * **Retraining is an engineering optimization, not a guess**: The fitted staleness cost function $(T^* \approx \sqrt{2C/(Q \cdot V \cdot \text{Accuracy}_0 \cdot \gamma)})$ connects retraining frequency to quantitative economics under its stated assumptions.
 * **Deploy through graduated rollout with pretested rollback**: Canary, blue-green, and shadow deployments match risk profiles, with tiered rollback strategies that must be tested regularly through fire drills.
 * **Stage the investment**: Monitoring and continuous integration/deployment are often the first investments. High-value or high-risk systems justify more rigor than low-impact internal analytics. Add feature stores when training-serving skew becomes measurable; add automated retraining as the model matures.
-* **The five principles transfer across domains**: Reproducibility (version artifacts), Separation of Concerns (modular layers), Consistency (validated training-serving parity), Observable Degradation (layered monitoring), and Cost-Aware Automation (retraining economics). Domain constraints change how each principle is implemented and which infrastructure supports it.
+* **The five principles transfer across domains**: Reproducibility, separation of concerns, consistency, observable degradation, and cost-aware automation remain stable; domain constraints change their implementation and supporting infrastructure.
 * **Operational maturity is staged and organizational**: Managing one model differs qualitatively from managing many. The principles scale, but complexity grows with fleet size, and shared on-call rotations and unified incentives are as critical as tooling.
 
 :::

book/quarto/contents/vol1/ml_systems/ml_systems.qmd

@@ -4371,7 +4371,7 @@ class AmdahlCameraCalc:
 
 **Fallacy**: *Model optimization translates linearly to system speedup.*
 
-Amdahl's Law[^fn-amdahls-law-pipeline]\index{Amdahl's law!speedup limits}\index{Optimization!Amdahl's law} establishes hard limits that the bottleneck principle (@sec-ml-systems-bottleneck-principle-3514) makes operational, where @sec-appdx-machine-foundations-strong-scaling-amdahls-law-c6c2 derives the strong-scaling form and works a speedup example at eight processors: $\text{Speedup}_{\text{overall}} = \frac{1}{(1-p) + \frac{p}{s}}$ where $p$ is the fraction of work that can be improved and $s$ is the speedup of that fraction. Consider tapping the shutter on a smartphone camera. The image passes through `{python} AmdahlCameraCalc.cam_isp_ms_str` of signal processing (auto-exposure, white balance), `{python} AmdahlCameraCalc.cam_ml_ms_str` of ML scene classification, and `{python} AmdahlCameraCalc.cam_post_ms_str` of postprocessing (tone mapping, HDR merge)---`{python} AmdahlCameraCalc.cam_total_ms_str` total. Optimizing the ML classifier to run `{python} AmdahlCameraCalc.cam_ml_speedup_mult_str` faster (`{python} AmdahlCameraCalc.cam_ml_opt_ms_str` instead of `{python} AmdahlCameraCalc.cam_ml_ms_str`) drops total time from `{python} AmdahlCameraCalc.cam_total_ms_str` to `{python} AmdahlCameraCalc.cam_total_opt_ms_str`---only `{python} AmdahlCameraCalc.cam_speedup_10x_mult_str` overall, not `{python} AmdahlCameraCalc.cam_ml_speedup_mult_str`. Even eliminating ML entirely $(s = \infty)$ achieves only `{python} AmdahlCameraCalc.cam_speedup_inf_mult_str` speedup, because the remaining `{python} AmdahlCameraCalc.cam_non_ml_pct_str` of the pipeline is untouched. Effective optimization requires profiling the entire pipeline and addressing bottlenecks systematically, because system performance depends on the slowest unoptimized stage.
+Amdahl's Law[^fn-amdahls-law-pipeline]\index{Amdahl's law!speedup limits}\index{Optimization!Amdahl's law} establishes hard limits that the bottleneck principle (@sec-ml-systems-bottleneck-principle-3514) makes operational, where @sec-appdx-machine-foundations-strong-scaling-amdahls-law-c6c2 derives the strong-scaling form and works a speedup example at eight processors: $\text{Speedup}_{\text{overall}} = \frac{1}{(1-p) + \frac{p}{s}}$ where $p$ is the fraction of work that can be improved and $s$ is the speedup of that fraction. Consider tapping the shutter on a smartphone camera. The image passes through `{python} AmdahlCameraCalc.cam_isp_ms_str` of signal processing (auto-exposure, white balance), `{python} AmdahlCameraCalc.cam_ml_ms_str` of ML scene classification, and `{python} AmdahlCameraCalc.cam_post_ms_str` of postprocessing (tone mapping, HDR merge)---`{python} AmdahlCameraCalc.cam_total_ms_str` total. Optimizing the ML classifier to run `{python} AmdahlCameraCalc.cam_ml_speedup_mult_str` faster (`{python} AmdahlCameraCalc.cam_ml_opt_ms_str` instead of `{python} AmdahlCameraCalc.cam_ml_ms_str`) drops total time from `{python} AmdahlCameraCalc.cam_total_ms_str` to \mbox{`{python} AmdahlCameraCalc.cam_total_opt_ms_str`}---only `{python} AmdahlCameraCalc.cam_speedup_10x_mult_str` overall, not `{python} AmdahlCameraCalc.cam_ml_speedup_mult_str`. Even eliminating ML entirely $(s = \infty)$ achieves only `{python} AmdahlCameraCalc.cam_speedup_inf_mult_str` speedup, because the remaining `{python} AmdahlCameraCalc.cam_non_ml_pct_str` of the pipeline is untouched. Effective optimization requires profiling the entire pipeline and addressing bottlenecks systematically, because system performance depends on the slowest unoptimized stage.
 
 [^fn-amdahls-law-pipeline]: [offset=45mm] **Amdahl's law**\index{Amdahl's law!pipeline bottleneck}: Formalized by @amdahl1967 for multiprocessor scaling, this principle applies directly to ML deployment pipelines where the model is only one stage among many. In this camera pipeline example, ML inference is `{python} AmdahlCameraCalc.cam_ml_ms_str` of `{python} AmdahlCameraCalc.cam_total_ms_str`; even a 100$\times$ model speedup yields only about `{python} AmdahlCameraCalc.range_speedup_mult_str` end-to-end improvement because the rest of the pipeline is unchanged. Teams that benchmark model latency in isolation systematically overestimate deployment gains.