Interesting Concept but Doesn't Work

[ChadwickTheCrab]

Experience Report: ESP32-S3 N16R8 + RuView Real-World Deployment

Hardware

• 5x AITRIP ESP32-S3 N16R8 (ESP32-S3-WROOM-1 module, 16MB flash, 8MB Octal PSRAM, no display)
• Connected via native USB-JTAG (/dev/ttyACM0) and FT232 UART (/dev/ttyUSB0)
• WiFi: Standard 2.4 GHz AP (SSID: Blose)

What We Learned

1. Pre-built firmware binaries do NOT work on N16R8 boards

The release binaries at firmware/esp32-csi-node/release_bins/ produce boards that:

• Connect to WiFi successfully
• Boot into the adaptive controller
• Immediately enter state 8 (ADAPT_STATE_DEGRADED) and never leave
• Only send 60-byte feature-state heartbeat packets (type 0x06) — no CSI frames ever
• CSI callback count stays at 0 permanently — pkt_yield_per_sec is 0, so the degraded gate at adaptive_controller_decide.c:27 (obs->pkt_yield_per_sec < cfg->min_pkt_yield) immediately locks the board in DEGRADED state with no recovery path

2. Building firmware from source fixes it

Building the firmware using the IDF Docker container:

cd firmware/esp32-csi-node
docker run --rm -v "$(pwd):/project" -w /project \
  espressif/idf:v5.4 bash -c \
  'rm -rf build sdkconfig && idf.py set-target esp32s3 && idf.py build'

Self-built firmware boots to state 6 (ADAPT_STATE_SENSE_ACTIVE) with real CSI frames streaming at 30-40 fps. Boards successfully enter sensing states and respond to motion/presence.

3. Flashing considerations

• --flash_mode dio --flash_size 8MB worked fine for 16MB flash boards — the partition table only uses ~7MB so the extra space is unused
• The eFuse on these boards reports Flash type: quad (4 data lines) but DIO mode still functions
• Must flash: bootloader.bin, partition-table.bin, ota_data_initial.bin, esp32-csi-node.bin at 0x20000
• Provision with --edge-tier 2 --force-partial

4. Docker sensing server works well

• ruvnet/wifi-densepose:latest receives and processes real ESP32 CSI data
• Use --disable-host-validation for LAN access
• Multi-node auto-detection works — nodes appear as they come online
• WebSocket at :3001/ws/sensing and REST API at :3000/api/v1/sensing/latest both reflect real data

5. Model loading

• The wifi-densepose-v1.rvf in the docker/ directory loads but produces ~20K+ engine errors and gets trust-demoted
• HuggingFace model at ruvnet/wifi-densepose-pretrained can be converted to RVF via --convert-model (after fixing safetensors header null-padding issue), but still gets trust-demoted with hundreds of engine errors
• Person detection (n_persons) appears to use a heuristic, not the model — documented as "slot-capacity heuristic, not a learned classifier" in the firmware readme

6. Current observations

• 3 nodes streaming real CSI simultaneously
• Person detection state flickers between present/moving and still even with no one in the room (sleeping family on another floor)
• Adaptive controller shows motion=1.00 presence=3.79 in state 6 even in empty rooms
• The yield=0pps problem in DEGRADED state has no recovery mechanism — once a board enters state 8 it stays there permanently

Questions / Suggestions

1. Could the pre-built release binaries be rebuilt with the current sdkconfig? The ones in release_bins/ appear to have an incompatible config for boards with Octal PSRAM
2. The DEGRADED state should have a timeout or recovery path — if yield recovers above threshold, the board should leave state 8
3. Training documentation would be valuable — what does "empty room" mean for calibration? Do pets trigger false positives? How many minutes of calibration data are needed?
4. The trust/engine-error demotion system needs documentation — what causes demotion, how to diagnose, how to recover

Current Setup (working)

• 3/5 boards flashed with self-built firmware, provisioned via provision.py
• Docker server receiving UDP on port 5005, dashboard at port 3000
• HuggingFace model converted and loaded (but trust-demoted)
• All nodes send to 192.168.1.4:5005 with unique --node-id

[ruvnet]

The AITRIP ESP32-S3 N16R8 uses 16 MB flash and 8 MB octal PSRAM. The project's supported hardware is ESP32-S3 with 8 MB flash (N8R8 variant with quad-PSRAM). The pre-built binaries in firmware/esp32-csi-node/release_bins/ are compiled with CONFIG_ESPTOOLPY_FLASHSIZE_8MB and quad-PSRAM initialization — running them on an octal-PSRAM board causes PSRAM init to fail silently, which lets WiFi come up but kills the CSI data pipeline (hence 0 pps).

To get CSI data from N16R8 boards you'll need to build from source with matching config:

• CONFIG_ESPTOOLPY_FLASHSIZE_16MB=y
• CONFIG_SPIRAM_MODE_OCT=y (and the matching CONFIG_ESPTOOLPY_OCT_FLASH=y if your variant uses octal flash)

Tutorial #34 covers the full build process, and #301 has the known-issues/troubleshooting index. The compact-board heat warning in firmware/esp32-csi-node/README.md is also worth a read for sustained-radio-on workloads.

---

Generated by Claude Code

[ruvnet]

Thanks for the detailed report — this is exactly the kind of end-to-end deployment writeup that's most useful. Following up on the prior comment and going through each point:

1. The "doesn't work" root cause — resolved, as the earlier comment covers

Confirmed accurate: the pre-built release_bins/ are compiled for the quad-PSRAM N8R8 variant, and your N16R8 boards use octal PSRAM — that mismatch silently breaks PSRAM init, which is why WiFi came up fine but CSI never flowed (permanent 0 pps). Building from source with CONFIG_SPIRAM_MODE_OCT=y (and matching flash config) is the correct fix, and matches what you found. #34 and #301 have the build/troubleshooting detail.

2. "DEGRADED state has no recovery path" — this isn't actually a design gap

I checked adaptive_controller_decide.c directly: the degraded-gate check (pkt_yield_per_sec < min_pkt_yield) re-evaluates on every single decision tick, not just once on entry. If yield recovers above the threshold, the state machine does leave DEGRADED on the next tick — there's no latch or one-way trap in the code.

What actually happened on your original boards: the root cause in #1 (wrong PSRAM config) meant pkt_yield_per_sec was stuck at literal 0 forever, because the CSI pipeline itself never started — not because the state machine refused to re-check. Once you rebuilt with the correct config and yield became nonzero, the same gate logic would clear DEGRADED normally. So no firmware change needed here — just wanted to close the loop on why it looked stuck.

3. "Person detection flickers present/moving in an empty room, even with real CSI streaming" — this was a real bug, now fixed

This matched a pattern several other reports converged on, and it turned out to be two separate, real bugs in the sensing-server, both found, fixed, and merged since your report:

• sensing-server: classification emits presence=false with motion_level=present_moving and confidence=1.0 — UIs gate on presence, so a moving person renders as an empty room #1442 — the server's classification output could say motion_level: "present_moving" and presence: false in the same frame (internally contradictory), and since the UI gates on presence, that produced exactly what you'd see as "flickering" between present and empty. Fixed in PR #1447.
• Controlled reproduction: raw-CSI presence detection sits at chance (AUC ≈ 0.58) once time-of-day is controlled — the two apparent successes were confounds #1440 — deeper investigation (a separate, very rigorous controlled report) found the per-room presence calibration could learn a threshold that reads a genuinely empty room as "present" on every single frame, when a calm/still occupant's calibration sample happened to show less signal variance than the room's own ambient noise floor. Fixed in PR #1448.

Both are merged to main. Given you're running real hardware, if you get a chance to re-run against current main I'd be interested whether the flickering is gone — that's not a demand, just an offer if you're still set up.

4. Training/calibration docs, and trust/engine-error demotion docs — still genuinely open

You're right that both are thin right now. Rather than let this catch-all issue stay open indefinitely for unrelated follow-up work, I've opened a dedicated tracking issue scoped specifically to the two documentation gaps: what "empty room" means for calibration (duration, pets, HVAC), and how the trust/engine-error demotion system works and how to diagnose/recover from it (including the wifi-densepose-v1.rvf / converted-HF-model trust-demotion you hit) — see #1456.

---

Closing this one since the core "doesn't work" complaint has a confirmed root cause and fix path, the DEGRADED-state concern turned out not to be a real gap, and the presence-flickering you observed matches bugs that are now fixed on main. The remaining documentation work continues in #1456. Really appreciate the thoroughness here — this is a model bug report.