Merge pull request #1048 from ruvnet/fix/issues-1031-894-fusion-guard-model-load

fix: multistatic fusion guard for real TDM (#1031) + load published HF model via auto-detect/convert (#894)

Author: ruvnet

CHANGELOG.md

@@ -7,6 +7,10 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [Unreleased]
 
+### Fixed
+- **Multistatic fusion guard was too tight for real TDM hardware (#1031).** `MultistaticConfig::default().guard_interval_us` was 5,000 µs (5 ms) with a comment claiming "well within the 50 ms TDMA cycle" — but on a real N-slot TDM schedule node `k` transmits in slot `k`, so two nodes are separated by the *slot offset*, not clock jitter. A real 2-node mesh (slots 0/1) measured an **18,194 µs** spread, so every real frame set exceeded the 5 ms guard and `fuse()` silently fell back to per-node sum/dedup — multistatic fusion never actually ran on hardware. Raised the default hard guard to **60 ms** (a full 50 ms TDMA cycle + 20% jitter headroom, derived from the slot model and documented in the field doc) and the soft guard to **20 ms** (just above the observed 18.2 ms 2-slot spread, so a normal cycle fuses cleanly with no privacy demotion). Added `MultistaticConfig::for_tdm_schedule(total_slots, slot_duration_us)` to derive the guard from a deployment's exact schedule, and a `WDP_TDM_SLOTS`+`WDP_TDM_SLOT_US` env seam in sensing-server. The honest per-node fallback remains for genuinely-mismatched frames — now the exception, not the default. Pinned by `fuse_real_tdm_spread_18194us_fuses_with_default_guard` (fails on the old 5 ms default) + `configurable_guard_rejects_too_large_spread` (guard still rejects a spread beyond one cycle).
+- **Published HuggingFace model was unloadable — RVF format mismatch (#894).** The `ProgressiveLoader` rejected the published `ruvnet/wifi-densepose-pretrained` model with the opaque `invalid magic at offset 0: expected 0x52564653 (RVFS), got 0x77455735`, then silently fell back to signal heuristics (the "10 persons for 1" garbage reporters saw). The HF repo ships `model.safetensors`, `model-q{2,4,8}.bin` (magic `0x77455735` = "5WEw"), and `model.rvf.jsonl` — none carry the binary-RVF magic. New `model_format` module **auto-detects** RVFS / safetensors / HF-quant-bin / JSONL by magic+name, returns a **typed actionable** `ModelLoadError` (lists accepted formats + the one-command convert path — never the opaque magic), and **converts** `model.safetensors` / `model.rvf.jsonl` → RVF in-memory so the published full-precision model now loads via `--model`. A `--convert-model <in> --convert-out <out>` CLI subcommand gives a one-command offline path; the silent heuristics fallback is now a loud, actionable error. **Honest scope:** the converter wires the format/load path (safetensors F32 tensors → RVF weight segment, manifest written, Layer A/B/C all succeed, weights round-trip) — it does **not** claim end-to-end pose accuracy, since the HF pose-decoder architecture differs from this crate's inference head (still data-gated in #894). Quantized `.bin` blobs are rejected with a typed error pointing at the safetensors path. Pinned by `safetensors_converts_and_loads` + `hf_quant_classifies_to_actionable_error` (both fail on the old opaque-magic path).
+
 ### Changed
 - **Mesh partition risk now demotes the privacy class and is witnessed (ADR-032).** The dynamic min-cut guard's `at_risk` signal was advisory-only (it fed the recalibration advisor). It now also contributes to the ADR-141 privacy demotion alongside fusion- and array-level contradictions: a mesh close to partitioning makes the fused belief less trustworthy, so the cycle emits at a more restricted class (monotonic — information only removed). Because `effective_class` feeds the BLAKE3 witness, a fragmenting array now shifts the witness — partition risk is auditable, not just logged. The mesh computation moved ahead of the demotion step in `process_cycle`; new `mesh_guard_mut()` exposes risk-threshold tuning. Test proves a forced-risk 3-node cycle demotes PrivateHome Anonymous→Restricted and shifts the witness vs a clean *same-topology* baseline (the only delta between the two cycles is the forced risk).
 

v2/crates/wifi-densepose-engine/src/lib.rs

@@ -682,8 +682,9 @@ mod tests {
     fn contradiction_demotes_privacy() {
         let (mut e, room) = engine();
         let cal = CalibrationId(7);
-        // 2 ms spread: within the 5 ms hard guard but above the 1 ms soft guard.
-        let frames = [node_frame(0, 1000, 56), node_frame(1, 3000, 56)];
+        // 25 ms spread: within the 60 ms hard guard but above the 20 ms soft
+        // guard (#1031 raised both to accommodate the real TDM slot offset).
+        let frames = [node_frame(0, 1_000, 56), node_frame(1, 26_000, 56)];
         let out = e.process_cycle(&frames, cal, room, 20_000).unwrap();
 
         assert!(out.demoted, "loose alignment must demote");

v2/crates/wifi-densepose-sensing-server/src/lib.rs

@@ -17,6 +17,7 @@ pub mod graph_transformer;
 pub mod host_validation;
 pub mod introspection;
 pub mod matter;
+pub mod model_format;
 pub mod mqtt;
 pub mod path_safety;
 pub mod semantic;

v2/crates/wifi-densepose-sensing-server/src/main.rs

@@ -14,6 +14,7 @@ pub mod cli;
 pub mod csi;
 mod engine_bridge;
 mod field_bridge;
+mod model_format;
 mod multistatic_bridge;
 pub mod pose;
 mod rvf_container;
@@ -144,6 +145,16 @@ struct Args {
     #[arg(long, value_name = "PATH")]
     export_rvf: Option<PathBuf>,
 
+    /// Convert a published model file (model.safetensors / model.rvf.jsonl) to
+    /// the RVF binary container the --model loader expects, then exit (#894).
+    /// Pair with --convert-out for the destination path.
+    #[arg(long, value_name = "PATH")]
+    convert_model: Option<PathBuf>,
+
+    /// Output path for --convert-model (defaults to <input>.rvf).
+    #[arg(long, value_name = "PATH")]
+    convert_out: Option<PathBuf>,
+
     /// Run training mode (train a model and exit)
     #[arg(long)]
     train: bool,
@@ -6221,6 +6232,34 @@ fn vitals_snapshots_from_sensing_json(
     }
 }
 
+/// Build the multistatic guard config, optionally derived from the TDM schedule
+/// declared in the environment (#1031).
+///
+/// When both `WDP_TDM_SLOTS` and `WDP_TDM_SLOT_US` parse as positive integers,
+/// the guard is derived via [`MultistaticConfig::for_tdm_schedule`] so a
+/// deployment can match its exact schedule. Otherwise the published default
+/// (60 ms hard / 20 ms soft) is returned. `min_nodes` is *not* set here — the
+/// caller overrides it for single-node passthrough.
+fn multistatic_guard_config_from_env() -> MultistaticConfig {
+    multistatic_guard_config_from(
+        std::env::var("WDP_TDM_SLOTS").ok().as_deref(),
+        std::env::var("WDP_TDM_SLOT_US").ok().as_deref(),
+    )
+}
+
+/// Pure core of [`multistatic_guard_config_from_env`] for testability.
+fn multistatic_guard_config_from(slots: Option<&str>, slot_us: Option<&str>) -> MultistaticConfig {
+    match (
+        slots.and_then(|s| s.trim().parse::<usize>().ok()),
+        slot_us.and_then(|s| s.trim().parse::<u64>().ok()),
+    ) {
+        (Some(n), Some(us)) if n >= 1 && us >= 1 => {
+            MultistaticConfig::for_tdm_schedule(n, us)
+        }
+        _ => MultistaticConfig::default(),
+    }
+}
+
 /// Turn a `ProgressiveLoader::new` failure into an actionable diagnostic (#894).
 ///
 /// The published HuggingFace `ruvnet/wifi-densepose-pretrained` files
@@ -6230,6 +6269,11 @@ fn vitals_snapshots_from_sensing_json(
 /// `0x52564653`). Feeding one to `--model` produced a bare
 /// "invalid magic at offset 0 …" that left users stuck. Detect the common
 /// cases and explain plainly what's loadable instead.
+///
+/// Superseded in the live load path by [`load_or_convert_model`] (which now
+/// converts the convertible formats instead of just explaining), but retained
+/// as the human-readable format-landscape summary and exercised by tests.
+#[allow(dead_code)]
 fn diagnose_model_load_error(path: &std::path::Path, data: &[u8], err: &str) -> String {
     let name = path
         .file_name()
@@ -6270,6 +6314,124 @@ fn diagnose_model_load_error(path: &std::path::Path, data: &[u8], err: &str) ->
     )
 }
 
+/// Load a model for `--model`, auto-detecting + converting the published
+/// HuggingFace formats when the native RVF loader rejects them (issue #894).
+///
+/// Order of operations:
+/// 1. Try the native RVF `ProgressiveLoader` (the only format with `RVFS` magic).
+/// 2. On failure, **auto-detect** the format. If it is convertible
+///    (`safetensors` / `model.rvf.jsonl`), convert it in-memory to RVF and load
+///    that — so the published `model.safetensors` becomes loadable here.
+/// 3. If it is a non-convertible format (quantized blob / unknown), return the
+///    typed, actionable [`model_format::ModelLoadError`] message — never the
+///    opaque "invalid magic …" string.
+///
+/// Returns the loaded `ProgressiveLoader` or a human-actionable error string.
+fn load_or_convert_model(
+    path: &std::path::Path,
+    data: &[u8],
+) -> Result<ProgressiveLoader, String> {
+    use model_format::{convert_to_rvf, detect_format, ModelFormat};
+
+    // 1. Native RVF.
+    if let Ok(loader) = ProgressiveLoader::new(data) {
+        return Ok(loader);
+    }
+
+    let name = path
+        .file_name()
+        .and_then(|n| n.to_str())
+        .unwrap_or("")
+        .to_string();
+    let model_id = path
+        .file_stem()
+        .and_then(|s| s.to_str())
+        .unwrap_or("converted-model");
+
+    match detect_format(data, &name) {
+        // 2. Convertible formats: convert in-memory, then load.
+        ModelFormat::Safetensors | ModelFormat::JsonlManifest => {
+            match convert_to_rvf(data, &name, model_id) {
+                Ok(rvf_bytes) => {
+                    info!(
+                        "Model `{}` is {} — converting to RVF in-memory and loading (issue #894)",
+                        path.display(),
+                        detect_format(data, &name).label()
+                    );
+                    ProgressiveLoader::new(&rvf_bytes).map_err(|e| {
+                        format!(
+                            "converted {} to RVF but the container failed to load: {e}",
+                            detect_format(data, &name).label()
+                        )
+                    })
+                }
+                Err(conv_err) => Err(conv_err.to_string()),
+            }
+        }
+        // 3. Non-convertible: typed actionable error.
+        _ => Err(model_format::classify_load_failure(
+            data,
+            &name,
+            "RVF container parse failed",
+        )
+        .to_string()),
+    }
+}
+
+/// `--convert-model` entry point (issue #894): read `in_path`, convert it to an
+/// RVF binary container, write it to `out_path`, and verify the result loads.
+/// Returns a process exit code (0 = success).
+fn run_convert_model(in_path: &std::path::Path, out_path: &std::path::Path) -> i32 {
+    let data = match std::fs::read(in_path) {
+        Ok(d) => d,
+        Err(e) => {
+            eprintln!("convert-model: failed to read {}: {e}", in_path.display());
+            return 1;
+        }
+    };
+    let name = in_path
+        .file_name()
+        .and_then(|n| n.to_str())
+        .unwrap_or("")
+        .to_string();
+    let model_id = in_path
+        .file_stem()
+        .and_then(|s| s.to_str())
+        .unwrap_or("converted-model");
+
+    let detected = model_format::detect_format(&data, &name);
+    eprintln!(
+        "convert-model: detected {} ({} bytes)",
+        detected.label(),
+        data.len()
+    );
+
+    match model_format::convert_to_rvf(&data, &name, model_id) {
+        Ok(rvf_bytes) => {
+            // Verify the converted bytes actually load before writing.
+            if let Err(e) = ProgressiveLoader::new(&rvf_bytes) {
+                eprintln!("convert-model: produced RVF did NOT load (bug): {e}");
+                return 1;
+            }
+            if let Err(e) = std::fs::write(out_path, &rvf_bytes) {
+                eprintln!("convert-model: failed to write {}: {e}", out_path.display());
+                return 1;
+            }
+            eprintln!(
+                "convert-model: wrote {} ({} bytes). Load it with `--model {}`.",
+                out_path.display(),
+                rvf_bytes.len(),
+                out_path.display()
+            );
+            0
+        }
+        Err(e) => {
+            eprintln!("convert-model: {e}");
+            1
+        }
+    }
+}
+
 /// Whether `--export-rvf` should emit the placeholder container-format demo.
 ///
 /// It must only do so **standalone**. Combined with `--train`/`--pretrain` the
@@ -6323,6 +6485,17 @@ async fn main() {
         return;
     }
 
+    // Handle --convert-model: turn a published HF model file (safetensors /
+    // model.rvf.jsonl) into the RVF binary container --model expects, then exit
+    // (issue #894). Gives the reporter a one-command path off the heuristics.
+    if let Some(ref in_path) = args.convert_model {
+        let out_path = args
+            .convert_out
+            .clone()
+            .unwrap_or_else(|| in_path.with_extension("rvf"));
+        std::process::exit(run_convert_model(in_path, &out_path));
+    }
+
     // Handle --export-rvf: writes a CONTAINER-FORMAT DEMO with placeholder
     // weights — it is NOT a trained model. Only short-circuit when standalone:
     // combined with --train/--pretrain the real model is exported by the
@@ -6951,7 +7124,7 @@ async fn main() {
         if args.progressive || args.model.is_some() {
             info!("Loading trained model (progressive) from {}", mp.display());
             match std::fs::read(mp) {
-                Ok(data) => match ProgressiveLoader::new(&data) {
+                Ok(data) => match load_or_convert_model(mp, &data) {
                     Ok(mut loader) => {
                         if let Ok(la) = loader.load_layer_a() {
                             info!(
@@ -6963,7 +7136,13 @@ async fn main() {
                         progressive_loader = Some(loader);
                     }
                     Err(e) => {
-                        error!("{}", diagnose_model_load_error(mp, &data, &e.to_string()))
+                        // #894: typed, actionable message (never the opaque magic)
+                        // and a LOUD warning that we are degrading to heuristics.
+                        error!("{e}");
+                        error!(
+                            "Model NOT loaded — falling back to signal heuristics. \
+                             Pose/person-count output will be approximate (issue #894)."
+                        );
                     }
                 },
                 Err(e) => error!("Failed to read model file: {e}"),
@@ -7136,9 +7315,14 @@ async fn main() {
         pose_tracker: PoseTracker::new(),
         last_tracker_instant: None,
         multistatic_fuser: {
+            // #1031: the default guard (60 ms hard / 20 ms soft) accommodates a
+            // real TDM slot offset. A deployment can override it to match its
+            // own schedule via WDP_TDM_SLOTS + WDP_TDM_SLOT_US (both set ⇒ derive
+            // from the schedule), else the published default is used.
+            let cfg = multistatic_guard_config_from_env();
             let mut fuser = MultistaticFuser::with_config(MultistaticConfig {
                 min_nodes: 1, // single-node passthrough
-                ..Default::default()
+                ..cfg
             });
             if let Some(ref pos_str) = args.node_positions {
                 let positions = field_bridge::parse_node_positions(pos_str);