|
| 1 | +# Roadmap |
| 2 | + |
| 3 | +Items are not ordered by priority. Priority will be determined by integration |
| 4 | +requirements as the system matures. |
| 5 | + |
| 6 | +--- |
| 7 | + |
| 8 | +## System |
| 9 | + |
| 10 | +- **Package names and namespaces** — current naming is verbose. Shorter, consistent |
| 11 | + naming across packages, namespaces, and topics is planned. |
| 12 | +- **OCCT 8 transition** — currently built against OCCT 7.9.3. |
| 13 | + Transitioning to OCCT 8 is planned to take advantage of API improvements and |
| 14 | + long-term support. |
| 15 | +- **Test coverage** — unit and integration test coverage needs to be increased |
| 16 | + systematically across all packages. |
| 17 | +- **Behavior tree integration** — the coordinator is a temporary placeholder. |
| 18 | + Full behavior tree based task orchestration is the system-level end goal, |
| 19 | + replacing the coordinator and enabling conditional, multi-step, arbitrary-robot |
| 20 | + scenes. |
| 21 | +- **Mesh geometry support** — both the gripper sampler and planning pipeline have |
| 22 | + mesh support as independent goals. At the system level this is a unified capability |
| 23 | + milestone enabling sensor-driven workflows where CAD is unavailable. |
| 24 | +- **Real hardware validation** — current validation is limited to simulation and |
| 25 | + GP25 geometry. Systematic real hardware testing across supported configurations |
| 26 | + is planned. |
| 27 | +- **Mesh-based effectors and extra axes** — the system currently assumes parallel |
| 28 | + jaw grippers and fixed-base robots. Integrating more realistic mesh-based effectors |
| 29 | + and extended extra axis support is planned as part of the robot-agnostic extensibility |
| 30 | + goal. |
| 31 | + |
| 32 | +--- |
| 33 | + |
| 34 | +## hold_and_weld_gripper_sampler |
| 35 | + |
| 36 | +### Plugin System |
| 37 | + |
| 38 | +- Pluginize constraint system (`ExclusionZoneConstraint`, `KissingSurfaceConstraint`) |
| 39 | + via pluginlib. Base interfaces are already defined. |
| 40 | +- Pluginize filter system (`SurfaceFilter`, `RegionFilter`) via pluginlib. |
| 41 | + Base interfaces are already defined. |
| 42 | +- Orientation grader — currently quality score is grippable arc fraction only. |
| 43 | + Add policy choices: force closure metric, approach clearance, robot joint reachability. |
| 44 | + |
| 45 | +### Pipeline Improvements |
| 46 | + |
| 47 | +- Smart ground rejection. Current halfspace model is geometry agnostic. Replace with |
| 48 | + topology aware detection using workpiece surface normals and z-position analysis. |
| 49 | +- Concurrency. Pipeline is deliberately single threaded for proof of concept clarity. |
| 50 | + Contact pair processing and radial map construction are the primary parallelization |
| 51 | + targets. |
| 52 | +- Dynamic sampling. Several sampling parameters are currently static. Adaptive density |
| 53 | + based on surface geometry and gripper dimensions is planned. |
| 54 | +- OCCT pipeline can benefit from mesh support techniques to identify geometrically |
| 55 | + fertile sampling areas more efficiently, reducing brute-force surface traversal. |
| 56 | + |
| 57 | +### Validation |
| 58 | + |
| 59 | +- Integration testing for filters and exclusion zone constraints beyond current unit tests. |
| 60 | +- Organic shape testing. Current validation covers box, prism, and small cylinder only. |
| 61 | + Complex non-convex geometry is not yet validated. |
| 62 | + |
| 63 | +### Action Server Integration |
| 64 | + |
| 65 | +- Wire `hold_and_weld_gripper_sampler` into the `hold_and_weld_application` action server |
| 66 | + system via ROS 2 lifecycle action server. Reference implementations exist in |
| 67 | + `hold_and_weld_application`. |
| 68 | + |
| 69 | +### Mesh Support |
| 70 | + |
| 71 | +Mesh based geometry input as an alternative to CAD for sensor driven or CAD unavailable |
| 72 | +scenarios. Planned as a two-phase pipeline. |
| 73 | + |
| 74 | +Phase one performs structural analysis of the mesh — identifying graspable regions and |
| 75 | +natural surface boundaries through graph based traversal. A spanning tree based approach |
| 76 | +is used to guarantee complete mesh coverage and handle topologically circular regions |
| 77 | +where no natural boundary exists. CGAL is the targeted library given existing dependency |
| 78 | +availability in the system. |
| 79 | + |
| 80 | +Phase two generates ranked contact pairs through guided sampling, starting from |
| 81 | +geometrically promising regions and propagating outward via breadth-first search. Surface |
| 82 | +boundaries are classified as either geometric features or traversal constructs, kept |
| 83 | +strictly separate so ranking reflects only real geometry. Output feeds into the existing |
| 84 | +contact pair interface with no downstream changes required. |
| 85 | + |
| 86 | +This is a significant architectural addition planned after core CAD pipeline stabilization. |
| 87 | + |
| 88 | +### Dataset Generation |
| 89 | + |
| 90 | +An output pipeline that takes mesh inputs and produces labeled grasp datasets. The |
| 91 | +practical motivation is that weld areas, screw holes, and similar features tend to repeat |
| 92 | +across workpieces in the same workplace. A generated dataset allows downstream models to |
| 93 | +learn avoidance of these regions without rerunning the full sampling pipeline on every |
| 94 | +new workpiece. This would also enable gripping strategies informed by spatial proximity |
| 95 | +to weld seams. |
| 96 | + |
| 97 | +### Code Quality |
| 98 | + |
| 99 | +- Add `BRepCheck_Analyzer` validation at geometry entry points for robustness on external |
| 100 | + STEP files. |
| 101 | +- Fix known error handling inconsistencies per error handling policy. Primarily single |
| 102 | + layer catch blocks in `contact_point_sampler.cpp` and unguarded `UIso`/`VIso` handle |
| 103 | + checks in `shape_refiner.cpp`. |
| 104 | + |
| 105 | +--- |
| 106 | + |
| 107 | +## hold_and_weld_application |
| 108 | + |
| 109 | +### Action Servers |
| 110 | + |
| 111 | +- Worker thread watchdog timeout. |
| 112 | +- `detach_object` wiring in `run_job` once re-grasp workflow is defined. |
| 113 | +- ACM collision allowance per object instead of per link — required to handle complex |
| 114 | + multi-primitive objects correctly where per-link granularity is insufficient. |
| 115 | +- `YAML` value validation in `load_job_from_yaml`. |
| 116 | +- Online parameter update for execution-time tunable parameters: velocity scaling, |
| 117 | + acceleration scaling, controller type. |
| 118 | +- Replace `move_to_pose` with proper implementation. |
| 119 | +- Limit extra axis movement by parameters. |
| 120 | +- Planning scene coordinator to make scene management event driven. |
| 121 | + |
| 122 | +### Coordinator |
| 123 | + |
| 124 | +- Multi-planner pipeline: OMPL for collision-free approach planning, Pilz Industrial |
| 125 | + Motion Planner for sharp deterministic path execution — cleaner separation of concerns |
| 126 | + than the current single-planner approach. |
| 127 | +- Deprecate coordinator in favor of behavior tree integration. |
| 128 | +- Scene management via behavior tree nodes. |
| 129 | + |
| 130 | +### Kinematics |
| 131 | + |
| 132 | +- More test coverage on viable and non-viable seam paths. |
| 133 | +- Lifecycle harmony with MoveIt 2 — currently requires architectural workarounds due to |
| 134 | + MoveIt 2's internal nodes not accepting lifecycle node interfaces. |
| 135 | +- Approach configuration finder — derive a guaranteed-valid joint goal from the |
| 136 | + validator's IK walk, feed directly to OMPL as goal state, eliminating the current |
| 137 | + retry loop entirely. |
| 138 | + |
| 139 | +--- |
| 140 | + |
| 141 | +## hold_and_weld_planning |
| 142 | + |
| 143 | +The OCCT pipeline is largely stable. Deterministic seam extraction, normal evaluation, |
| 144 | +and pose generation work reliably for the tested joint types. Pipe joint detection |
| 145 | +remains incomplete and is not planned for the near term. |
| 146 | + |
| 147 | +The mesh pipeline has one fundamental open problem: CGAL corefinement produces |
| 148 | +intersection segments whose endpoints are pinned to triangle edges rather than the true |
| 149 | +geometric intersection curve. On simple geometry this approximation is acceptable. On |
| 150 | +complex or organic shapes the scatter becomes significant enough to corrupt path |
| 151 | +classification and normal extraction downstream. |
| 152 | + |
| 153 | +The candidate approaches to fix this without introducing learned components are: |
| 154 | + |
| 155 | +- **Newton refinement** — project each corefinement vertex onto the true intersection |
| 156 | + by iteratively solving the two-tangent-plane system using per-vertex normals from both |
| 157 | + meshes. Eliminates tessellation pinning at the cost of one closest-point query per |
| 158 | + vertex per iteration. |
| 159 | +- **Ridge-valley detection on the edge-contributing mesh** — since all practical weld |
| 160 | + joints involve at least one geometric edge, the intersection region always has a |
| 161 | + curvature discontinuity on at least one mesh. Detecting that ridge directly may give |
| 162 | + a cleaner curve than post-processing corefinement output. |
| 163 | + |
| 164 | +Neither approach is implemented. Both remain as research directions pending |
| 165 | +re-engagement with the mesh pipeline. |
| 166 | + |
| 167 | +A new core primitive is planned for complex curves that cannot be adequately represented |
| 168 | +as sequences of line and arc segments. This will support point-by-point pose generation |
| 169 | +directly from the raw seam geometry, allowing organic and irregular intersection shapes |
| 170 | +to be planned without requiring classification into geometric primitives. |
| 171 | + |
| 172 | +A separate planned capability is seam extraction from scanned mesh inputs where CAD |
| 173 | +geometry is unavailable. This requires a different pipeline: plane segmentation from the |
| 174 | +mesh, intersection computation between segmented planes, and convexity-based validation |
| 175 | +to distinguish weld joints from internal mesh edges. This would extend the system to |
| 176 | +sensor-driven workflows and real-world scanned geometry where the current CAD-dependent |
| 177 | +pipelines cannot be used. |
0 commit comments