Skip to content

Commit 09ca660

Browse files
ruvnetruvnet
committed
docs(adr): plan RTL8720F radar SDK integration
1 parent c7b3f0b commit 09ca660

3 files changed

Lines changed: 306 additions & 0 deletions

File tree

‎docs/adr/ADR-263-rtl8720f-2-4ghz-fmcw-radar-platform.md‎

Lines changed: 161 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,161 @@
1+
# ADR-263: Adopt RTL8720F 2.4 GHz FMCW radar as an optional RuView sensing platform
2+
3+
- **Status**: proposed
4+
- **Date**: 2026-07-18
5+
- **Deciders**: ruv
6+
- **Tags**: realtek, rtl8720f, ameba, fmcw, radar, cfr, csi, hardware
7+
- **Relates to**: ADR-018, ADR-063, ADR-064, ADR-095, ADR-097, ADR-260, ADR-262
8+
9+
## Context
10+
11+
Realtek's `RTL8720F-2.4G-Radar-Advantages_EN.pptx` describes an RTL8720F mode that shares the
12+
2.4 GHz radio between Wi-Fi, Bluetooth, and an active FMCW radar. It offers two data products that
13+
are useful to RuView:
14+
15+
1. **CFR (Channel Frequency Report)**, described by Realtek as the same concept as Wi-Fi CSI.
16+
2. **Near and far Range-FFT reports**, preserving near-field content while extending observation to
17+
approximately 5–6 m.
18+
19+
The proposed radio uses one transmit and one receive antenna, 20/40/70 MHz sweeps, configurable
20+
8/16/32/64 microsecond chirp symbols, a maximum 2.56 ms FMCW packet, and a configurable frame
21+
interval above 15 ms. The deck recommends 40 MHz outside Japan and 20 MHz in Japan. It also
22+
describes EDCCA/CTS channel access, Wi-Fi/BT/radar time division, interference reporting, and
23+
priority arbitration in the driver.
24+
25+
This is not a drop-in replacement for ESP32 CSI:
26+
27+
- it is **active monostatic FMCW**, while the ESP32 path observes Wi-Fi packet CSI;
28+
- one Tx/one Rx has no angle-of-arrival or native multi-target separation;
29+
- the stated 40 MHz range resolution is about 3.15 m, despite a finer 0.59 m Range-FFT report step;
30+
- the presentation is a capability description, not an SDK contract. It contains no header names,
31+
function signatures, callback ABI, binary layouts, toolchain version, licensing terms, or public
32+
RTL8720F board package.
33+
34+
Realtek's public Ameba RTOS repository is a plausible base and its public release history includes
35+
CSI and EDCCA APIs, but the reviewed public material does not establish that the RTL8720F radar
36+
API or firmware blobs are publicly available. Therefore an honest integration must be split at a
37+
vendor boundary.
38+
39+
## Decision
40+
41+
RuView will support RTL8720F radar as an **optional, capability-negotiated source**, without
42+
replacing the ESP32 firmware or treating radar CFR as byte-compatible with ADR-018 CSI.
43+
44+
The integration has three layers:
45+
46+
1. **Realtek device firmware**: a small application built in the vendor-supported Ameba SDK calls
47+
the radar API, owns coexistence configuration, and emits versioned reports. This code lives under
48+
`firmware/rtl8720f-radar/` only after the redistributable SDK/API is available.
49+
2. **Transport-neutral wire contract**: CFR and Range-FFT reports are framed independently from the
50+
vendor ABI and sent over UDP, USB CDC, or UART. ADR-264 defines this boundary.
51+
3. **Rust host adapter**: `wifi-densepose-hardware` parses reports from bytes and converts CFR into
52+
the existing CSI-domain representation, while Range-FFT remains a radar modality and feeds the
53+
RuField/RuView cross-modality bridge from ADR-260/262.
54+
55+
The two report types remain semantically distinct:
56+
57+
| RTL8720F output | RuView representation | Permitted use |
58+
|---|---|---|
59+
| CFR | `CsiFrame` through a Realtek calibration adapter | CSI feature extraction after validation |
60+
| Range-FFT near/far | `RadarFrame` / RuField `mmwave_radar`-class event with a 2.4 GHz descriptor | range, motion, presence, fusion |
61+
| Vendor AI presence probability | derived observation with model/version provenance | advisory input, never ground truth |
62+
| Interference report | quality/provenance metadata | reject, down-weight, or mark contaminated frames |
63+
64+
The modality registry should eventually distinguish `fmcw_radar_2_4ghz` from `mmwave_radar`; until
65+
that RuField schema revision is accepted, the adapter must attach `carrier_hz = 2.4e9` and must not
66+
claim millimetre-wave provenance.
67+
68+
## Delivery phases and gates
69+
70+
### P0 — Vendor enablement
71+
72+
Obtain a redistributable RTL8720F SDK package, radar API headers/libraries, a supported evaluation
73+
board, flashing/debug instructions, report definitions, and written redistribution terms.
74+
75+
**Gate:** compile and run Realtek's unmodified radar example and capture CFR plus near/far
76+
Range-FFT output. Until this passes, device firmware is `VENDOR_BLOCKED`, not implemented.
77+
78+
### P1 — Host-first contract
79+
80+
Implement ADR-264 types, parsers, fixtures, fuzz tests, and replay support without linking vendor
81+
code. Generate deterministic synthetic fixtures whose provenance is explicitly synthetic.
82+
83+
**Gate:** malformed inputs never panic; encode/decode round trips; unknown versions and report
84+
types fail closed.
85+
86+
### P2 — RTL8720F firmware adapter
87+
88+
Wrap only the minimum vendor API surface: initialization, profile configuration, start/stop,
89+
callback acquisition, interference status, and report serialization. Keep vendor types out of the
90+
wire protocol.
91+
92+
**Gate:** 30-minute simultaneous Wi-Fi telemetry and radar capture with no watchdog reset, bounded
93+
loss, monotonic sequence numbers, and explicit coexistence/interference statistics.
94+
95+
### P3 — Calibration and signal validation
96+
97+
Calibrate CFR phase/amplitude, Range-FFT bin spacing, static leakage, and clock drift. Compare
98+
reported range against measured targets at multiple distances and bandwidths.
99+
100+
**Gate:** publish measured error distributions. Do not infer accuracy from report-bin spacing and
101+
do not advertise multi-person pose or vital signs from the vendor deck.
102+
103+
### P4 — Fusion and productization
104+
105+
Feed calibrated CFR through the CSI path and Range-FFT through RuField, retaining source, mode,
106+
bandwidth, calibration, firmware, and interference provenance.
107+
108+
**Gate:** ablation shows whether the radar stream improves a named RuView metric over ESP32 CSI
109+
alone. If it does not, ship it only as an independent presence/range sensor.
110+
111+
## Consequences
112+
113+
### Positive
114+
115+
- One low-cost radio can provide active radar and CSI-like CFR while retaining Wi-Fi connectivity.
116+
- Range-FFT adds an independent physical measurement for presence/range fusion.
117+
- The vendor SDK is isolated from the Rust sensing core and from the stable on-wire contract.
118+
- Capability negotiation permits future Realtek parts without another application-level fork.
119+
120+
### Negative
121+
122+
- The first implementation is blocked on access to the actual RTL8720F radar SDK/API and hardware.
123+
- Active 2.4 GHz transmission changes coexistence, privacy, power, and regional compliance concerns.
124+
- 1T1R and limited sweep bandwidth cannot provide the spatial resolution of multi-antenna mmWave.
125+
- A second embedded toolchain and firmware release process must be maintained.
126+
127+
### Neutral
128+
129+
- ESP32 remains the default CSI node.
130+
- Existing consumers receive normalized frames and do not link against Realtek code.
131+
- Vendor AI output is optional metadata; RuView retains responsibility for its own validation.
132+
133+
## Rejected alternatives
134+
135+
1. **Map Range-FFT directly to `CsiFrame`.** Rejected because range bins and channel-frequency
136+
samples have different axes and physical meaning.
137+
2. **Link the Realtek SDK into the Rust server.** Rejected because it couples host builds to a
138+
proprietary embedded ABI and toolchain.
139+
3. **Wait to define any interface until hardware arrives.** Rejected because the host protocol,
140+
parser safety, replay, and provenance can be developed and reviewed independently.
141+
4. **Replace ESP32 nodes.** Rejected because the modes are complementary and availability differs.
142+
143+
## Open vendor questions
144+
145+
- Exact RTL8720F part/board identifier and production availability.
146+
- SDK repository/tag, compiler, RTOS, binary blobs, license, and redistribution permissions.
147+
- Radar initialization/configuration/callback API signatures and threading/ISR constraints.
148+
- CFR and near/far Range-FFT element type, complex ordering, scaling, endianness, and timestamps.
149+
- Whether CFR is calibrated complex data and whether phase remains coherent across frames.
150+
- Maximum report rates, buffer ownership, DMA/cache constraints, and Wi-Fi throughput impact.
151+
- Region/channel enforcement and whether 70 MHz operation is allowed by the supplied firmware.
152+
- Secure boot, signed OTA, unique device identity, and firmware attestation support.
153+
154+
## Sources
155+
156+
- Realtek Semiconductor, `RTL8720F-2.4G-Radar-Advantages_EN.pptx`, slides 3 and 10–19,
157+
supplied 2026-07-18. This is product material, not measured RuView validation.
158+
- [Ameba-AIoT/ameba-rtos releases](https://github.com/Ameba-AIoT/ameba-rtos/releases), reviewed
159+
2026-07-18; public release notes mention CSI and EDCCA APIs but do not document the deck's radar ABI.
160+
- ADR-063 (mmWave sensor fusion), ADR-095/097 (source normalization), and ADR-260/262 (RuField
161+
multimodal event model and live bridge).

‎docs/adr/ADR-264-rtl8720f-radar-wire-protocol.md‎

Lines changed: 140 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,140 @@
1+
# ADR-264: Versioned wire protocol for RTL8720F CFR and Range-FFT reports
2+
3+
- **Status**: proposed
4+
- **Date**: 2026-07-18
5+
- **Deciders**: ruv
6+
- **Tags**: realtek, rtl8720f, protocol, cfr, range-fft, udp, serial
7+
- **Depends on**: ADR-263
8+
- **Relates to**: ADR-018, ADR-095, ADR-097, ADR-099, ADR-260
9+
10+
## Context
11+
12+
ADR-263 adopts RTL8720F radar behind an anti-corruption boundary. The Realtek presentation names
13+
CFR, near Range-FFT, far Range-FFT, and interference reports, but does not specify their binary ABI.
14+
RuView needs a stable, testable contract that can be implemented before the vendor SDK arrives and
15+
that will not expose vendor structs, pointer layouts, padding, or callback lifetime rules over the
16+
network.
17+
18+
ADR-018 already defines ESP32 CSI framing. Reusing its magic or pretending that Realtek radar is an
19+
ESP32 packet would make source detection ambiguous and erase radar-specific calibration metadata.
20+
21+
## Decision
22+
23+
Define a new little-endian `RtlRadarFrameV1` envelope with its own magic and explicit payload type.
24+
This is a RuView protocol, not a claim about Realtek's native memory layout.
25+
26+
### Envelope
27+
28+
All integer fields are little-endian. Floating-point payloads use IEEE-754 binary32. No C struct is
29+
sent by `memcpy`; firmware serializes each field explicitly.
30+
31+
| Offset | Size | Field | Meaning |
32+
|---:|---:|---|---|
33+
| 0 | 4 | magic | ASCII `RTR1` (`0x31525452`) |
34+
| 4 | 1 | version | `1` |
35+
| 5 | 1 | report_type | 1 CFR, 2 range-near, 3 range-far, 4 interference, 5 capabilities |
36+
| 6 | 2 | header_len | complete header size, initially 56 |
37+
| 8 | 4 | frame_len | header + payload + CRC |
38+
| 12 | 4 | sequence | wraps modulo 2^32 |
39+
| 16 | 8 | timestamp_us | monotonic device time at acquisition |
40+
| 24 | 8 | device_id | stable pseudonymous identifier, not a MAC address |
41+
| 32 | 4 | center_freq_khz | RF centre frequency |
42+
| 36 | 2 | bandwidth_mhz | 20, 40, or 70 |
43+
| 38 | 2 | flags | calibration/interference/saturation/time-sync flags |
44+
| 40 | 2 | element_count | complex samples or range bins |
45+
| 42 | 1 | element_format | 1 complex-i16, 2 complex-f32, 3 power-u16, 4 power-f32 |
46+
| 43 | 1 | antenna_count | expected to be 1 for the deck's 1T1R configuration |
47+
| 44 | 4 | scale | quantized-to-physical multiplier; `1.0` for float payloads |
48+
| 48 | 4 | bin_spacing | Hz for CFR, metres for Range-FFT |
49+
| 52 | 4 | calibration_id | device calibration revision/hash prefix |
50+
| 56 | variable | payload | determined by type, count, and format |
51+
| final-4 | 4 | crc32 | IEEE CRC-32 over header and payload |
52+
53+
If vendor evidence shows that 56 bytes is too costly, a later protocol version may introduce a
54+
compact header. V1 favors auditable provenance over premature byte savings.
55+
56+
### Payload semantics
57+
58+
- **CFR** contains ordered complex channel-frequency samples. The adapter must know the frequency
59+
origin/order and must not fabricate missing phase. Uncalibrated frames carry the uncalibrated flag
60+
and cannot enter phase-sensitive processing.
61+
- **Range-near/range-far** contains ordered range bins. Near and far are separate report types so
62+
filtering and leakage behavior are never hidden from consumers.
63+
- **Interference** contains a versioned TLV set for channel-busy, detected-during-chirp, estimated
64+
interference power, and packet jitter. Unknown TLVs are skipped by length.
65+
- **Capabilities** is emitted at boot and on request. It declares supported report types, bandwidths,
66+
chirp lengths, maximum elements/report, maximum frame rate, firmware version, and SDK identifier.
67+
68+
### Transport
69+
70+
The identical envelope is supported over:
71+
72+
- UDP datagrams for normal RuView ingestion;
73+
- USB CDC or UART with COBS framing and a zero-byte delimiter;
74+
- file replay as a length-prefixed sequence of envelopes.
75+
76+
One envelope must fit one UDP datagram. Fragmentation is not part of V1; firmware rejects a profile
77+
whose maximum report exceeds the configured MTU and reports the required size through capabilities.
78+
79+
### Parser and trust rules
80+
81+
The host parser:
82+
83+
1. validates magic, version, lengths, enum values, element count/format multiplication, and CRC
84+
before allocating or decoding the payload;
85+
2. caps frames at 64 KiB and elements at a configured hardware maximum;
86+
3. rejects non-finite float metadata/payload values;
87+
4. tracks sequence gaps and timestamp regressions per device;
88+
5. preserves unknown flags but never interprets them as trusted;
89+
6. attaches transport source, firmware/SDK version, calibration ID, and interference state to
90+
provenance;
91+
7. labels fixture/generated frames as synthetic.
92+
93+
No vendor-provided presence probability bypasses RuView privacy, provenance, or quality gates.
94+
95+
## Consequences
96+
97+
### Positive
98+
99+
- Firmware, transport, parser, replay, and fusion can evolve independently.
100+
- Fuzzing and golden fixtures require no Realtek SDK or board.
101+
- CFR and Range-FFT retain correct axes and calibration provenance.
102+
- A boot-time capabilities frame makes SDK/API drift observable.
103+
104+
### Negative
105+
106+
- Serialization adds CPU and bandwidth overhead compared with dumping a vendor buffer.
107+
- V1 fields may need revision after the actual API and report limits are disclosed.
108+
- UDP provides integrity/error detection, not authenticity or confidentiality.
109+
110+
### Neutral
111+
112+
- Authentication can be layered with ADR-032 device identity or a signed RuField receipt without
113+
changing report semantics.
114+
- ESP32 ADR-018 framing remains unchanged.
115+
116+
## Implementation plan
117+
118+
1. Add `rtl8720f` types/parser module to `wifi-densepose-hardware` behind no vendor dependency.
119+
2. Add golden CFR, near/far Range-FFT, interference, and capabilities fixtures.
120+
3. Add property/fuzz tests for length arithmetic, enum handling, CRC, and float validation.
121+
4. Add a replay CLI that prints normalized metadata without running inference.
122+
5. Once SDK access exists, implement the embedded serializer and verify captured frames against the
123+
host golden decoder.
124+
6. Revise this proposed ADR with measured element counts, rates, and API names before acceptance.
125+
126+
## Acceptance criteria
127+
128+
- Rust encode/decode round-trip for every report type.
129+
- Cross-language golden vector produced by the RTL8720F firmware.
130+
- Zero parser panics over the fuzz corpus and arbitrary byte input.
131+
- Detection of single-bit corruption, truncation, count overflow, timestamp regression, and gaps.
132+
- Captured CFR frequency order and Range-FFT bin spacing verified against vendor documentation and a
133+
measured target.
134+
135+
## Sources
136+
137+
- Realtek Semiconductor, `RTL8720F-2.4G-Radar-Advantages_EN.pptx`, slides 11–19, supplied
138+
2026-07-18.
139+
- ADR-018 (ESP32 framing), ADR-095/097 (hardware normalization), ADR-260 (multimodal event model),
140+
and ADR-263 (platform decision).

‎docs/adr/README.md‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,10 @@
11
# Architecture Decision Records
22

3+
Latest proposed decisions:
4+
5+
- [ADR-264: Versioned wire protocol for RTL8720F CFR and Range-FFT reports](ADR-264-rtl8720f-radar-wire-protocol.md)
6+
- [ADR-263: Adopt RTL8720F 2.4 GHz FMCW radar as an optional RuView sensing platform](ADR-263-rtl8720f-2-4ghz-fmcw-radar-platform.md)
7+
38
This folder contains 182 Architecture Decision Records (ADRs) that document every significant technical choice in the RuView / WiFi-DensePose project. (The index tables below list a curated subset per domain; see the directory listing for the full set.)
49

510
## Why ADRs?

0 commit comments

Comments
 (0)