Add SMALL_VGICP Koide manifest for Phase 1 backend comparison#93
Add SMALL_VGICP Koide manifest for Phase 1 backend comparison#93rsasaki0109 wants to merge 3 commits into
Conversation
Phase 1 compares NDT_OMP vs SMALL_GICP vs SMALL_VGICP on Koide outdoor_hard_01. The NDT_OMP and SMALL_GICP manifests already exist; the SMALL_VGICP one was missing even though the backend is fully wired (registration_backend_policy.hpp + component, vgicp_voxel_resolution). This mirrors the SMALL_GICP manifest exactly except registration_method (SMALL_VGICP), the output dir, and the ROS domain id, so the two small_gicp variants differ only in the registration type. small_gicp is installed and the Koide data is local, so the three-way comparison is turn-key; it only needs an idle machine (load < ~5) for valid alignment-time numbers.
The _full_gt_diag_map manifests reference data not on disk; the runnable Koide data is outdoor_hard_01a + map_outdoor_hard.ply. Add 01a smoke manifests for the three backends and document the first 60s result in docs/phase1_koide_backend_comparison.md. Key finding: translation-RMSE alone is a trap. With full-density Livox scans SMALL_GICP/SMALL_VGICP are throughput-bound (1.5-3.4 s/align vs NDT 0.15 s) and produce only ~10-15 poses, so their low RMSE is a mirage. With identical downsampling (voxel 0.5) SMALL_GICP becomes the accuracy/stability winner (0.095 m, 100% ok, 0.40 s/align) while NDT_OMP is the fastest baseline (0.266 m, 89% ok) and SMALL_VGICP is not viable at vgicp_voxel_resolution 0.5 (38.9% ok). Single 60s smoke; full 380s and a VGICP resolution sweep are follow-ups.
|
Ran the comparison on the locally-available outdoor_hard_01a (the Key finding: translation-RMSE alone is a trap. With full-density Livox scans SMALL_GICP/SMALL_VGICP are throughput-bound (1.5–3.4 s/align vs NDT 0.15 s) → only ~10–15 poses → their low RMSE is a mirage over the seed neighbourhood. Fair comparison with identical downsampling (voxel 0.5):
Single 60 s smoke; full 380 s and a VGICP resolution sweep are follow-ups. Details in |
…t winner Ran the full outdoor_hard_01a window (380s). The 60s-smoke ranking is overturned: NDT_OMP 0.668m vs SMALL_GICP 11.70m vs SMALL_VGICP 22.68m. SMALL_GICP looked best on the easy first 60s (0.095m, 100% ok) but is fragile -- it loses lock on the hard section and never recovers (lost 274.5s of 380, fitness explodes to 1.3M, crop starves), diverging to 11.7m. NDT_OMP is less accurate while locked but survives the hard section (recovery_retry recovers once) and ends at 0.668m. A textbook case of why a single easy window is necessary-but-not-sufficient (the Phase 3 lesson). Adds the full380 manifests and the reversal section to the comparison doc.
|
Full 380 s window — the ranking REVERSES.
SMALL_GICP looked best on the easy first 60 s (0.095 m, 100% ok) but is fragile: on the hard section it loses lock and never recovers (lost 274.5 s of 380, fitness explodes to 1.3M, crop starves), diverging to 11.7 m. NDT_OMP is the robust winner (0.668 m) — less accurate while locked, but it survives the hard section (recovery_retry recovers once). The 60 s smoke ranking was exactly reversed by the full replay — a textbook case of why a single easy window is necessary-but-not-sufficient (the Phase 3 lesson). Verdict: NDT_OMP recommended default. SMALL_GICP = higher locked-in accuracy but not safe without lock-retention work; SMALL_VGICP not competitive as configured. Full details in |
1 participant
Adds the missing third manifest for the Phase 1 backend comparison (NDT_OMP vs SMALL_GICP vs SMALL_VGICP) on Koide outdoor_hard_01.
The NDT_OMP and SMALL_GICP manifests already exist; SMALL_VGICP was missing even though the backend is fully wired (registration_backend_policy.hpp maps it, the component constructs it with vgicp_voxel_resolution). This manifest mirrors the SMALL_GICP one exactly except registration_method, the output dir, and the ROS domain id, so the two small_gicp variants differ only in registration type.
small_gicp is installed (/usr/local/lib/libsmall_gicp.so) and the Koide data is local, so the three-way comparison is turn-key — it only needs an idle machine (load < ~5) for valid alignment-time numbers.
Config only; no code change.