You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
***The Systems & Infrastructure Engineer Path:** Focus on the memory and execution runtime: Part I (Trajectories, ACB) $\rightarrow$ Part II (Context & Radix Prefix Caching) $\rightarrow$ Part III (MCP Protocols, MicroVM Sandboxes, Sagas) $\rightarrow$ Part IV (Observability).
100
+
***The AI Researcher & Algorithm Architect Path:** Focus on search, planning, and error dynamics: Part I (Trajectories, State Graphs & DAGs, Scheduling) $\rightarrow$ Part II (Compounding Error & Test-Time Compute, Scratchpads & Isolation) $\rightarrow$ Part IV (Trajectory Benchmarks).
101
+
***The Enterprise AI Architect & Security Lead Path:** Focus on governance, isolation, and economics: Part I (Invariant Closure, Capability Leases) $\rightarrow$ Part III (Typed Contracts, Blast Radius & Sandboxing) $\rightarrow$ Part IV (Replay Debugging, Fleet Economics).
Copy file name to clipboardExpand all lines: books/vol3/frontmatter/author_note.qmd
+1-1Lines changed: 1 addition & 1 deletion
Original file line number
Diff line number
Diff line change
@@ -28,7 +28,7 @@ In *Introduction to Machine Learning Systems*, we optimized the single node: bal
28
28
29
29
Volume III confronts what happens when that assumption shatters.
30
30
31
-
Autonomy demands long horizons, but long horizons destroy reliability. If a model is ninety-eight percent accurate on any individual step, an unverified fifty-step autonomous workflow will fail nearly two-thirds of the time. Systems engineering is the only discipline that arrests that compounding decay. We must learn to manage context not as a bottomless prompt buffer, but as a structured memory hierarchy; to treat tool execution not as text strings, but as isolated, sandboxed system calls; and to build distributed Sagas with deterministic rollbacks when non-deterministic actions fail in the wild.
31
+
Autonomy demands long horizons, but long horizons destroy reliability. If a model is 98 percent accurate on any individual step, an unverified fifty-step autonomous workflow will fail nearly two-thirds of the time. Systems engineering is the only discipline that arrests that compounding decay. We must learn to manage context not as a bottomless prompt buffer, but as a structured memory hierarchy; to treat tool execution not as text strings, but as isolated, sandboxed system calls; and to build distributed Sagas with deterministic rollbacks when non-deterministic actions fail in the wild.
32
32
33
33
I write these chapters as a student of a discipline in motion. Putting this book together is my own way—as an educator and researcher—of wrestling with where the field is heading, sifting through the daily noise of transient frameworks, and boiling down the durable first principles of **Horizon, State, and Authority** that will still govern computing a decade from now.
Copy file name to clipboardExpand all lines: books/vol4/appendix/cross-disciplinary-lexicon.qmd
+8-4Lines changed: 8 additions & 4 deletions
Original file line number
Diff line number
Diff line change
@@ -27,11 +27,13 @@ Because each discipline developed its own vocabulary over decades, engineers fro
27
27
28
28
## 2. False Friends: Words That Cause Catastrophic Misunderstandings
29
29
30
+
Engineering teams building Physical AI systems are drawn from distinct engineering traditions—deep learning, classical robotics, embedded systems, and functional safety. Because these disciplines evolved independently, they frequently use the same vocabulary to denote fundamentally incompatible concepts. These "false friends" represent subtle linguistic traps where an assumption from one field, when carried uncritically into another, leads directly to catastrophic hardware failures.
31
+
30
32
### "Deterministic"
31
33
***In Deep Learning:** Setting `torch.manual_seed(42)` and `cudnn.deterministic = True` so the matrix multiplication forward pass produces bit-identical tensor values across offline runs.
32
34
***In Embedded & Real-Time Systems:** Guaranteeing that every periodic task completes in hardware time $t \le T_{\text{deadline}}$ with bounded jitter ($<1\,\mu\text{s}$), regardless of CPU cache misses, interrupt storms, or bus contention. *An algorithm that always outputs the same tensor 10 milliseconds too late is non-deterministic in physical systems.*
33
35
34
-
### "Real-Time"
36
+
### "Real-time"
35
37
***In Web & Cloud Software:** A web service or UI that updates within $100\text{--}200\text{ ms}$ ("real-time chat", "real-time streaming"), where occasional delays cause dropped frames or buffering spinners without danger.
36
38
***In Physical AI Systems:** Hard deadline determinism where missing a single $1.0\text{ ms}$ execution deadline ($1000\text{ Hz}$) produces an inverter current loop runaway, mechanical resonance, or irreversible collision.
37
39
@@ -59,20 +61,22 @@ Because each discipline developed its own vocabulary over decades, engineers fro
59
61
60
62
## 3. Practical Translation Guides for Students
61
63
62
-
### If you are coming from Computer Science / Deep Learning:
64
+
Navigating the multi-disciplinary landscape of Physical AI requires recognizing the implicit assumptions of your home field and actively translating your engineering instincts to match physical realities. The following translation guides provide actionable principles tailored to students and practitioners entering Physical AI from specific foundational backgrounds.
65
+
66
+
### If you are coming from computer science / deep learning:
63
67
64
68
1.**Forget `malloc()` on the fast loop:** You cannot allocate dynamic memory or resize lists inside a $1000\text{ Hz}$ control loop. Heap fragmentation causes random multi-millisecond pauses that crash physical robots.
65
69
2.**Your model is an unprivileged proposer:** A neural network must never have direct write access to motor registers. It proposes intent; a deterministic mathematical filter running on bare metal permits, clamps, or rejects the command.
66
70
3.**Loss $= 0$ in simulation does not mean it works:** Physics engines approximate contact dynamics as linear complementarity problems (LCP). Real contact friction is non-smooth, stiction-dominated, and thermally variable.
67
71
4.**Bandwidth matters more than FLOPS:** Streaming multi-gigabyte neural weights from LPDDR5 DRAM on edge SoCs creates a memory wall ($t_{\text{stream}} = M_{\text{weights}} / B_{\text{mem}}$) that limits inference cadence, regardless of how many TFLOPS the NPU matrix core claims.
68
72
69
-
### If you are coming from Mechanical Engineering / Control Theory:
73
+
### If you are coming from mechanical engineering / control theory:
70
74
71
75
1.**Uncertainty is multimodal:** Classical Kalman filters assume unimodal Gaussian noise. Perception models in the real world face discrete semantic hypotheses (e.g., an object is either in front of or behind a partition), requiring multimodal policy representations (Diffusion / Mixture Density Networks).
72
76
2.**You cannot write down the world dynamics $f(\mathbf{x})$ for an unconstrained kitchen or highway:** Analytical ODEs work for steel linkages in a factory cell, but fail when interacting with deformable objects, varying lighting, and open-world human behavior. Deep models provide the semantic perception front-end that classical control lacks.
73
77
3.**Control Barrier Functions are the bridge:** You do not have to discard control theory when using deep learning. Instead, treat the learned policy as a nominal reference generator $\mathbf{u}_{\text{ref}}$ and use a Control Barrier Function (CBF-QP) as an active safety filter that minimally adjusts $\mathbf{u}$ only when boundary invariants are threatened.
74
78
75
-
### If you are coming from Embedded Systems / Electrical Engineering:
79
+
### If you are coming from embedded systems / electrical engineering:
76
80
77
81
1.**AI compute is dynamic and stochastic:** You cannot treat neural inference like a fixed-cycle DSP FIR filter. Execution time varies based on image content, caching, and thermal throttling. You must design asynchronous two-speed architectures with expiring leases to absorb timing jitter safely.
78
82
2.**Data provenance is as critical as voltage integrity:** Logging teleoperation data without tracking operator response delays ($340\text{ ms}$) or safety governor clamps poisons training pipelines via covariate shift.
Copy file name to clipboardExpand all lines: books/vol4/appendix/heterogeneous-soc-reference.qmd
+2-2Lines changed: 2 additions & 2 deletions
Original file line number
Diff line number
Diff line change
@@ -25,7 +25,7 @@ In an embodied machine, cognitive deliberation and mechanical actuation operate
25
25
26
26
## Shared Memory RPMSG Message Contract
27
27
28
-
The application processor transmits timestamped intent proposals to the real-time safety controller over non-blocking shared SRAM mailboxes using the Remote Processor Messaging (RPMSG) protocol. The shared message schema (`PhysicalAgent_IntentProposal_t`) enforces a packed binary layout across cores:
28
+
The application processor transmits timestamped intent proposals to the real-time safety controller over non-blocking shared SRAM mailboxes using the Remote Processor Messaging (RPMSG) protocol. The shared message schema (`PhysicalAgent_IntentProposal_t`), detailed in @tbl-ipc-intent-contract, enforces a packed binary layout across cores:
29
29
30
30
| Field Name | Type | Size | Physical Units & Description |
0 commit comments