Skip to content

Commit e719e5d

Browse files
committed
fix(vol4): clean display math equations and align table formats
1 parent 2b8628e commit e719e5d

10 files changed

Lines changed: 130 additions & 130 deletions

File tree

books/vol4/chapters/01-boundary/01-boundary.qmd

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -295,7 +295,7 @@ In computer systems engineering, developers often ask: *why cannot we run this p
295295
Therefore, every robust physical AI machine physically partitions its compute across heterogeneous silicon (@fig-01-dual-brain):
296296

297297
1. **The Cognitive Proposer (Linux MPU / Edge NPU):** Operates in unprivileged user space at a deliberative cadence ($10\text{--}50\text{ Hz}$). It executes high-capacity neural policies, multi-modal perception backbones, and trajectory generators. It possesses **zero direct electrical wiring to physical actuator registers**. It can only write candidate trajectory proposals into a shared SRAM mailbox or lock-free circular ring buffer across an inter-processor communication (IPC) channel.
298-
2. **The Real-Time Permission Referee (Bare-Metal / RTOS MCU):** Operates on isolated, dedicated silicon at a hard real-time cadence ($1000\text{ Hz}$, $1\text{ ms}$ deadline, $< $50\,\mu\text{s}$$ jitter) with static, zero-allocation memory (`malloc` is strictly forbidden at runtime). It holds **exclusive physical authority** over the PWM compare registers, digital-to-analog converters, and gate drivers. The MCU continuously checks physical safety invariants and stopping clearance. If the proposed command satisfies all physical constraints, the MCU latches the values to hardware; if the proposal is unsafe, stalls, or panics, the MCU vetoes the command and executes a deterministic physical fallback.
298+
2. **The Real-Time Permission Referee (Bare-Metal / RTOS MCU):** Operates on isolated, dedicated silicon at a hard real-time cadence ($1000\text{ Hz}$, $1\text{ ms}$ deadline, $$< 50\,\mu\text{s}$ jitter) with static, zero-allocation memory (`malloc` is strictly forbidden at runtime). It holds **exclusive physical authority** over the PWM compare registers, digital-to-analog converters, and gate drivers. The MCU continuously checks physical safety invariants and stopping clearance. If the proposed command satisfies all physical constraints, the MCU latches the values to hardware; if the proposal is unsafe, stalls, or panics, the MCU vetoes the command and executes a deterministic physical fallback.
299299

300300
To bridge the asynchronous Linux OS and deterministic bare-metal firmware without priority inversion or lock contention, the MPU and MCU communicate across a shared SRAM mailbox using lock-free atomic sequence locks (*seqlocks*) or double-buffered ping-pong ring buffers.[^fn-hw-soc-mailbox] The MPU publishes candidate action chunks with a monotonically incrementing sequence version; the MCU reads the trajectory atomically without ever blocking on a mutex. If an MPU core stalls or experiences an OS scheduling delay, the sequence version fails to update, immediately signaling the MCU that proposed commands have aged beyond their freshness deadline.
301301

@@ -432,7 +432,7 @@ The next chapter (@sec-body) transitions from conceptual boundaries to physical
432432

433433
[^fn-hw-tim-pwm]: Power inverter timers enforce dead-time insertion ($50\text{--}500\text{ ns}$) to prevent shoot-through short circuits across DC buses. On ARM Cortex-M (`TIM1`/`TIM8`), capture/compare registers `TIMx_CCR` and preload register `TIMx_CCMR` buffer duty cycles until counter overflow to eliminate pulse glitches, mirrored in TI C2000 `ePWM` shadow registers.
434434

435-
[^fn-hw-serdes-bus]: Automotive SerDes links (TI FPD-Link III, ADI GMSL2/3) serialize multi-gigabit MIPI CSI-2 sensor streams over coaxial cables with sub-$15\text{ }\mu\text{s}$ transfer latency, hardware CRC-32 verification, and bidirectional backchannel `FSYNC` triggers that align multi-camera exposure start times within sub-microsecond jitter.
435+
[^fn-hw-serdes-bus]: Automotive SerDes links (TI FPD-Link III, ADI GMSL2/3) serialize multi-gigabit MIPI CSI-2 sensor streams over coaxial cables with sub-$15\,\mu\text{s}$ transfer latency, hardware CRC-32 verification, and bidirectional backchannel `FSYNC` triggers that align multi-camera exposure start times within sub-microsecond jitter.
436436

437437
[^fn-math-contact-lcp]: Rigid contact obeys Signorini-Coulomb complementarity ($0 \le \delta \perp F_N \ge 0$). Real-time stacks approximate compliance via Kelvin-Voigt models to prevent force loop divergence.
438438

books/vol4/chapters/04-nervous/04-nervous.qmd

Lines changed: 37 additions & 37 deletions
Large diffs are not rendered by default.

books/vol4/chapters/08-perception/08-perception.qmd

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -187,7 +187,7 @@ Visual perception does not operate in isolation; it functions alongside an Inert
187187
The spatial relationship between the IMU reference frame $\mathcal{F}_{\text{imu}}$ and the camera reference frame $\mathcal{F}_{\text{cam}}$ is defined by the rigid extrinsic transformation $\mathbf{T}_{\text{imu}}^{\text{cam}} = (\mathbf{R}_{\text{imu}}^{\text{cam}}, \mathbf{p}_{\text{imu}}^{\text{cam}})$. Because the camera and IMU are physically separated by a lever arm $\mathbf{p}_{\text{imu}}^{\text{cam}}$, an angular acceleration $\dot{\boldsymbol{\omega}}$ or angular velocity $\boldsymbol{\omega}$ experienced by the body induces an additional tangential and centripetal acceleration at the camera:
188188
$$\mathbf{a}_{\text{cam}} = \mathbf{R}_{\text{imu}}^{\text{cam}} (\mathbf{a}_{\text{imu}} - \mathbf{g}) + \boldsymbol{\omega} \times (\boldsymbol{\omega} \times \mathbf{p}_{\text{imu}}^{\text{cam}}) + \dot{\boldsymbol{\omega}} \times \mathbf{p}_{\text{imu}}^{\text{cam}}$$
189189

190-
Equally critical is the **temporal calibration offset** $\tau_{\text{cam-imu}} = t_{\text{cam}} - t_{\text{imu}}$. Standard commercial crystal oscillators on independent sensor breakout boards drift by roughly $\pm 50\text{ ppm}$, accumulating $$50\,\mu\text{s}$$ of error every second ($4.3\text{ s}$ per day). Moreover, analog anti-aliasing filters in the IMU, electronic exposure delays in the camera, and DMA driver scheduling inject fixed and variable temporal offsets between the two data streams.
190+
Equally critical is the **temporal calibration offset** $\tau_{\text{cam-imu}} = t_{\text{cam}} - t_{\text{imu}}$. Standard commercial crystal oscillators on independent sensor breakout boards drift by roughly $\pm 50\text{ ppm}$, accumulating $50\,\mu\text{s}$ of error every second ($4.3\text{ s}$ per day). Moreover, analog anti-aliasing filters in the IMU, electronic exposure delays in the camera, and DMA driver scheduling inject fixed and variable temporal offsets between the two data streams.
191191

192192
If the temporal offset $\Delta t$ is uncalibrated, platform rotation couples directly into catastrophic spatial estimation errors. Suppose a robot rotates at an angular velocity of $\omega = 2.0\text{ rad/s}$ ($115^\circ/\text{s}$). An undetected temporal lag of $\Delta t = 10\text{ ms}$ between the camera frame and the IMU integration causes a rotational orientation mismatch of:
193193
$$\Delta \theta = \omega \cdot \Delta t = 2.0\text{ rad/s} \times 0.010\text{ s} = 0.020\text{ rad} \approx 1.15^\circ$$
@@ -277,15 +277,15 @@ The maximum admissible ingestion load must therefore be derived from the enforce
277277
$$\Delta t_{\text{stall}} = \frac{V_{\text{burst}}}{R_{\text{arb}}} \le S$$
278278
Solving for the burst volume yields the maximum admissible ingestion load:
279279
$$V_{\text{burst}} \le S \times R_{\text{arb}}$$
280-
The admissible sensory payload is bounded by the smallest timing slack in the fastest enforcement reflex. In an analytical example where an enforcement monitor runs with a timing slack of $S = 2$50\,\mu\text{s}$$ and the memory controller delivers an effective service rate of $R_{\text{arb}} = 20.0\text{ GB/s}$ during write contention, the total sensory ingestion burst cannot exceed:
281-
$$V_{\text{burst}} \le 2$50\,\mu\text{s}$ \times 20.0\text{ GB/s} = 5.0\text{ MB}$$
280+
The admissible sensory payload is bounded by the smallest timing slack in the fastest enforcement reflex. In an analytical example where an enforcement monitor runs with a timing slack of $S = 250\,\mu\text{s}$ and the memory controller delivers an effective service rate of $R_{\text{arb}} = 20.0\text{ GB/s}$ during write contention, the total sensory ingestion burst cannot exceed:
281+
$$V_{\text{burst}} \le 250\,\mu\text{s} \times 20.0\text{ GB/s} = 5.0\text{ MB}$$
282282
Any sensor configuration whose combined DMA write burst exceeds $5.0\text{ MB}$ will stall the memory controller long enough to exhaust the slack, causing the safety monitor to miss its deadline regardless of the average bandwidth.
283283

284284
::: {#pri-ingress-bandwidth .callout-principle title="Ingress Bandwidth as a Safety Parameter"}
285285
Sensor resolution, frame rate, and channel count are safety parameters, not perception parameters; every megabyte of sensory DMA burst consumes memory bus service time directly from the safety refusal path.
286286
:::
287287

288-
This relationship establishes that sensor resolution, frame rate, and sensor count are safety parameters rather than purely perception parameters. In machine learning pipelines, engineering teams routinely increase camera resolution or add sensor modalities to improve open-loop model accuracy. In a physical machine, every additional pixel, bit of quantization depth, and exposure frame is an unavoidable withdrawal from the memory bandwidth budget that pays for timely refusal. Upgrading four cameras from $1080\text{p}$ ($1920 \times 1080$) to $4\text{K}$ ($3840 \times 2160$) quadruples the burst volume $V_{\text{burst}}$ from $3.11\text{ MB}$ to $12.44\text{ MB}$ per frame. On a system with the timing margins derived above, that single change pushes the worst-case memory stall from $155.5\,\mu\text{s}$ to $622.0\,\mu\text{s}$, exceeding the $2$50\,\mu\text{s}$$ slack and causing the safety enforcement loop to overrun its period. The higher-resolution image may produce a more confident detection in the learned model, but it removes the machine's guarantee of stopping in time.[^fn-std-iso-mpam]
288+
This relationship establishes that sensor resolution, frame rate, and sensor count are safety parameters rather than purely perception parameters. In machine learning pipelines, engineering teams routinely increase camera resolution or add sensor modalities to improve open-loop model accuracy. In a physical machine, every additional pixel, bit of quantization depth, and exposure frame is an unavoidable withdrawal from the memory bandwidth budget that pays for timely refusal. Upgrading four cameras from $1080\text{p}$ ($1920 \times 1080$) to $4\text{K}$ ($3840 \times 2160$) quadruples the burst volume $V_{\text{burst}}$ from $3.11\text{ MB}$ to $12.44\text{ MB}$ per frame. On a system with the timing margins derived above, that single change pushes the worst-case memory stall from $155.5\,\mu\text{s}$ to $622.0\,\mu\text{s}$, exceeding the $250\,\mu\text{s}$ slack and causing the safety enforcement loop to overrun its period. The higher-resolution image may produce a more confident detection in the learned model, but it removes the machine's guarantee of stopping in time.[^fn-std-iso-mpam]
289289

290290
The hardware mechanisms governing how frames move into memory, including PCIe scatter-gather descriptor rings, I/O memory management unit (IOMMU) page tables, and DRAM bank conflict schedulers, are standard computer systems topics treated in depth by @hennessy2019computer as well as @bryant2015computer. For the physical AI architect, the implementation details of descriptor rings sit below the floor. The quantity to budget is the latency penalty that sensory ingress inflicts on safety-critical memory transactions. When perception traffic claims the bus, the delay propagates directly into the physical domain. Every millisecond that memory contention steals from the enforcement loop is a millisecond during which the body continues along its existing trajectory without permission, converting a digital stall in the memory controller into an irrevocable loss of physical stopping distance.
291291
<!-- END -->
@@ -602,7 +602,7 @@ This requirement defines the handoff to @sec-memory. Where perception produces i
602602

603603
[^fn-hw-v4l2-dma]: Video4Linux2 (V4L2) kernel drivers use memory-mapped (`V4L2_MEMORY_MMAP`) or user-pointer (`V4L2_MEMORY_USERPTR`) I/O to allocate non-cacheable DMA scatter-gather buffers, streaming frames into contiguous physical memory without copying through kernel space.
604604

605-
[^fn-hw-tim-pwm]: Hardware timers (STM32 TIMx_CCR / Linux PTP PHC) latch exposure start times across camera GPIOs and IMU sampling registers simultaneously, bounding inter-sensor temporal skew to $<1\text{ }\mu\text{s}$.
605+
[^fn-hw-tim-pwm]: Hardware timers (STM32 TIMx_CCR / Linux PTP PHC) latch exposure start times across camera GPIOs and IMU sampling registers simultaneously, bounding inter-sensor temporal skew to $<1\,\mu\text{s}$.
606606

607607
[^fn-std-iso-mpam]: To bound memory access latency under DMA bursts, SoCs implement ARM MPAM (ISO 26262 ASIL D) and AXI QoS priority levels to ensure safety monitor reads preempt camera streams.
608608

books/vol4/chapters/10-intent/10-intent.qmd

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -355,7 +355,7 @@ A $32\text{-byte}$ cryptographic hash of the parent perception state binds the c
355355
| `0xA0..0xA7`<br>`terminal_flags` | `uint32_t` + `uint8_t[4]`<br>(8 bytes) | Bitmask enum + 4-byte zero-padding | $\text{Flag} \in \{\text{ACTIVE\_HOLD}, \text{COMPLIANT\_DROP}, \text{DECEL\_STOP}\}$ (deterministic fallback specification) | Hardware state machine selector; directly configures motor PWM gate drivers upon lease expiry | | | | |
356356
: **Intent Lease Protocol Contract Schema and Serialization Specification**: Byte-level memory layout ($168\text{ bytes}$ total), data types, physical coordinate frames, real-time validation invariants, and low-level nervous system enforcement mechanisms for the intent record. This strictly typed binary contract decouples high-level learned perception from low-level deterministic motor actuation, ensuring that unvalidated model hypotheses can never compromise physical hardware safety. {#tbl-10-intent-schema}
357357

358-
The nervous system evaluates monotonicity and provenance before passing an incoming intent record to the trajectory generator. When an asynchronous vision-language model requires $250\text{ ms}$ of inference time, a lighter geometric planner operating at $50\text{ Hz}$ may have already emitted a command derived from a newer camera exposure. By comparing the incoming sequence identifier against the highest sequence number committed to the actuation pipeline, the receiver discards out-of-order or replayed packets in constant time. If $t_{\text{src}}$ indicates that the perception frame is older than the current tracking epoch, or if the parent perception hash does not match an entry in the local sliding ring buffer of verified sensor frames, the packet is dropped immediately. This check executes in less than $1\text{ }\mu\text{s}$ of memory lookup and prevents the machine from executing delayed commands generated from stale visual scenes.
358+
The nervous system evaluates monotonicity and provenance before passing an incoming intent record to the trajectory generator. When an asynchronous vision-language model requires $250\text{ ms}$ of inference time, a lighter geometric planner operating at $50\text{ Hz}$ may have already emitted a command derived from a newer camera exposure. By comparing the incoming sequence identifier against the highest sequence number committed to the actuation pipeline, the receiver discards out-of-order or replayed packets in constant time. If $t_{\text{src}}$ indicates that the perception frame is older than the current tracking epoch, or if the parent perception hash does not match an entry in the local sliding ring buffer of verified sensor frames, the packet is dropped immediately. This check executes in less than $1\,\mu\text{s}$ of memory lookup and prevents the machine from executing delayed commands generated from stale visual scenes.
359359

360360
An intent record governs execution through a five-state finite state machine comprising the Pending, Active, Renewed, Expired, and Aborted states. Upon arrival over the real-time bus, the record enters the Pending state while the nervous system evaluates kinematic reachability and verifies that the commanded motion remains within the wrench limit $F_{\text{max}}$. Once validated, the record transitions to Active, granting the motor controllers authority to track toward the target pose $T$ within the tolerance envelope $\mathbf{\Sigma}$. If an upstream model generates a successor intent record with a higher sequence counter and valid provenance before the local hardware clock reaches $t_{\text{exp}}$, the state machine transitions to Renewed, updating the target pose and extending the expiration timestamp without interrupting joint velocity profiles. If the mechanism satisfies the target pose within the tolerance bounds $\mathbf{\Sigma}$, the lease completes; if joint torque sensors detect unexpected resistance exceeding $F_{\text{max}}$, the state machine transitions immediately to Aborted, cutting motor authority.
361361

0 commit comments

Comments
 (0)