Skip to content

Commit d9651ed

Browse files
committed
README changes, Add roadmap
Signed-off-by: silanus23 <berkantali23@outlook.com>
1 parent 29e704c commit d9651ed

8 files changed

Lines changed: 241 additions & 102 deletions

File tree

ADDING_A_ROBOT.md

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -104,8 +104,9 @@ Node(
104104
)
105105
```
106106

107-
**This single parameter is the entire connection between an action server and a
108-
specific physical robot.** Change it and the server plans for a different robot.
107+
**This parameter defines robot name groups for action servers.** Change it and the server plans for a different robot.
108+
109+
> **Note:** The welder action server currently has some hardcoded robot-specific values such as link names and tool frame references. This is a temporary compromise and will be parameterised in the same way as the gripper server. Adding a second welder before that work is done will require matching those hardcoded values to your robot.
109110
110111
### 6. Coordinator — connects to action servers by topic name
111112

README.md

Lines changed: 41 additions & 83 deletions
Original file line numberDiff line numberDiff line change
@@ -1,109 +1,67 @@
11
# Hold and Weld
22

3-
ROS 2 framework for automated dual-arm robotic welding systems that perform holding and welding simultaneously.
3+
**A ROS 2 framework for automated dual-arm robotic welding systems that perform holding and welding simultaneously with integrated grasp sampler and weld finder. Currently at PoC stage. Targets ROS 2 Jazzy.**
44

5-
## Overview
5+
![System Launch](docs/system_launch.GIF)
66

7-
A modular system for coordinating gripper and welding robots to autonomously manipulate and weld
8-
workpieces. This repository currently targets **ROS 2 Jazzy**.
9-
10-
## Features
7+
## Packages
118

12-
- Automated weld path generation from seam geometry and saving that as a json file
13-
- Dual-robot coordination with collision avoidance
14-
- Action-based architecture for flexible workflows
15-
- Support for multiple joint types (T-joint, butt-joint, corner joint, lap joint, edge joint)
9+
- [`hold_and_weld_description`](hold_and_weld_description) — robot URDFs, meshes, and models
10+
- [`hold_and_weld_planning`](hold_and_weld_planning) — weld path generation and seam extraction
11+
- [`hold_and_weld_gripper_sampler`](hold_and_weld_gripper_sampler) — constraint-aware grasp sampling
12+
- [`hold_and_weld_application`](hold_and_weld_application) — action servers and workflow coordination
13+
- [`hold_and_weld_bringup`](hold_and_weld_bringup) — launch files and system bringup
1614

1715
## Architecture
1816

1917
![System Architecture](docs/hold_and_weld.png)
2018

21-
## Packages
22-
23-
- **hold_and_weld_description** - Robot URDFs, meshes, and models
24-
- **hold_and_weld_planning** - Python tools for weld path generation and geometry processing
25-
- **hold_and_weld_application** - C++ action servers and workflow coordination
26-
27-
## Design Philosophy
28-
29-
This project is designed with the following principles in mind:
30-
31-
- **Separation of concerns**
32-
Planning, description, and application logic are split into independent packages to allow
33-
iteration and testing without cross-coupling.
34-
35-
- **Action-driven orchestration**
36-
Long-running and stateful robot operations are modeled as ROS 2 actions to support
37-
feedback, cancellation, and extensibility.
38-
39-
- **Simulation-first development**
40-
All core functionality is designed to be testable in simulation before deployment on real hardware.
41-
42-
- **Robot-agnostic extensibility**
43-
The system aims to support different robot models and configurations with minimal code changes,
44-
relying on URDF and configuration files rather than hardcoded assumptions.
45-
46-
- **Incremental autonomy**
47-
Advanced components such as geometry reasoning and behavior trees are introduced progressively,
48-
only after a stable execution pipeline is established.
49-
50-
51-
## Planned Milestones
52-
53-
### Milestone 1 – Core Stability
54-
- [ ] Realistic models for external-axis torch and gripper
55-
- [ ] Action server for moving a robot to a target pose
56-
- [ ] Utilize execute action messages instead only triggering ones.
57-
- [ ] Making task yaml monolithic and main source for every package
58-
- [ ] Replacing time based delay
59-
60-
### Milestone 2 – Configuration & Extensibility
61-
- [ ] Guidelines for adding robots and jobs
62-
- [ ] Test coverage for all supported weld joint types
63-
- [ ] Adding reachability test by adding jacobian calculations
64-
65-
### Milestone 3 – Advanced Autonomy
66-
- [ ] Behavior Tree–based task orchestration
67-
- [ ] URDF-driven geometry engine
19+
## Capabilities
20+
- Weld seam extraction from CAD and mesh geometry
21+
- Constraint-aware grasp sampling on exact CAD surfaces
22+
- Dual-arm coordination connecting both into an executable workcell
6823

69-
## Conceptual Design Notes (Future Work)
7024

71-
This section documents design considerations and architectural directions explored during development and is not a commitment to specific implementations.
25+
## Results
7226

27+
**Grasp candidates example generated by `hold_and_weld_gripper_sampler`:**
7328

74-
### Geometry Engine
29+
![Prism Grasp](docs/prism_grasp.png)
7530

76-
Currently the system can extract weld line coordinates and workpiece poses from URDFs alone.
77-
This iteration depends on defining weld geometry from URDFs and only calculates flat surfaces.
31+
**Weld seam extraction example via the mesh pipeline in `hold_and_weld_planning`:**
7832

79-
Future work includes making this geometry independent from joint types, choosing non-colliding
80-
orientations from multiple possibilities, extending to curved surfaces while maintaining
81-
work/travel angles, and being able to work with mesh files. Eventually this system should be
82-
capable of creating contemporary art installations with arbitrary weld geometries.
33+
![Cylinder Weld Finding](docs/cylinder_side.png)
8334

35+
## Quick Start
8436

85-
### Behavior Tree
37+
## Dependencies
38+
- [OpenCASCADE 7.9.3](https://github.com/Open-Cascade-SAS/OCCT/tree/OCCT-793)
39+
- CGAL
40+
- FCL
41+
- MoveIt 2
8642

87-
Currently there is a `dual_robot_coordinator` node that is responsible for orchestrating nodes.
88-
A behavior tree integration and making current action servers, node will help user to create
89-
multiple step, arbitrary number of robot and object scenes. On top of this a behavior tree can
90-
help user to create conditional check nodes to avoid failure situations of real life like a
91-
gripping failure mid mission.
43+
**2. Build the mesh intersection extension**
9244

93-
## Known Issues
45+
```bash
46+
cd hold_and_weld_planning/hold_and_weld_planning/mesh
47+
mkdir build && cd build
48+
cmake ..
49+
make
50+
cp mesh_intersection*.so ../
51+
```
9452

95-
- Currently gripper system is showing inconsistencies due to model problems.
96-
- JSON system can create problems that can be solved by building from cache after creating json. This problem
97-
will be solved by adding json parserer independent from colcon.
98-
- Current version is tested in in butt_joint and T-joint only more test cases from contributors are welcomed
99-
lap_joint's logic could be better in urdf generator.
53+
**3. Launch the system**
10054

101-
## License
55+
```bash
56+
ros2 launch hold_and_weld_bringup system_bringup.launch.py
57+
```
10258

103-
Apache-2.0
59+
For adding a new robot to the workcell see [ADDING_A_ROBOT.md](ADDING_A_ROBOT.md).
60+
For the development roadmap see [ROADMAP.md](ROADMAP.md).
10461

105-
## Author
62+
## References
10663

107-
Berkan Tali (berkantali23@outlook.com)
64+
Duff, T., Burgess, B., & Hery, C. (2017). Building an orthonormal basis, revisited.
65+
*Journal of Computer Graphics Techniques (JCGT)*, 6(1), 1–8.
10866

109-
GitHub: @silanus23
67+
Robot meshes from Yaskawa Motoman's [motoman_ros2_support_packages](https://github.com/ros-industrial/motoman) (Apache 2.0 / BSD 3-Clause). Thanks Yaskawa.

ROADMAP.md

Lines changed: 177 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,177 @@
1+
# Roadmap
2+
3+
Items are not ordered by priority. Priority will be determined by integration
4+
requirements as the system matures.
5+
6+
---
7+
8+
## System
9+
10+
- **Package names and namespaces** — current naming is verbose. Shorter, consistent
11+
naming across packages, namespaces, and topics is planned.
12+
- **OCCT 8 transition** — currently built against OCCT 7.9.3.
13+
Transitioning to OCCT 8 is planned to take advantage of API improvements and
14+
long-term support.
15+
- **Test coverage** — unit and integration test coverage needs to be increased
16+
systematically across all packages.
17+
- **Behavior tree integration** — the coordinator is a temporary placeholder.
18+
Full behavior tree based task orchestration is the system-level end goal,
19+
replacing the coordinator and enabling conditional, multi-step, arbitrary-robot
20+
scenes.
21+
- **Mesh geometry support** — both the gripper sampler and planning pipeline have
22+
mesh support as independent goals. At the system level this is a unified capability
23+
milestone enabling sensor-driven workflows where CAD is unavailable.
24+
- **Real hardware validation** — current validation is limited to simulation and
25+
GP25 geometry. Systematic real hardware testing across supported configurations
26+
is planned.
27+
- **Mesh-based effectors and extra axes** — the system currently assumes parallel
28+
jaw grippers and fixed-base robots. Integrating more realistic mesh-based effectors
29+
and extended extra axis support is planned as part of the robot-agnostic extensibility
30+
goal.
31+
32+
---
33+
34+
## hold_and_weld_gripper_sampler
35+
36+
### Plugin System
37+
38+
- Pluginize constraint system (`ExclusionZoneConstraint`, `KissingSurfaceConstraint`)
39+
via pluginlib. Base interfaces are already defined.
40+
- Pluginize filter system (`SurfaceFilter`, `RegionFilter`) via pluginlib.
41+
Base interfaces are already defined.
42+
- Orientation grader — currently quality score is grippable arc fraction only.
43+
Add policy choices: force closure metric, approach clearance, robot joint reachability.
44+
45+
### Pipeline Improvements
46+
47+
- Smart ground rejection. Current halfspace model is geometry agnostic. Replace with
48+
topology aware detection using workpiece surface normals and z-position analysis.
49+
- Concurrency. Pipeline is deliberately single threaded for proof of concept clarity.
50+
Contact pair processing and radial map construction are the primary parallelization
51+
targets.
52+
- Dynamic sampling. Several sampling parameters are currently static. Adaptive density
53+
based on surface geometry and gripper dimensions is planned.
54+
- OCCT pipeline can benefit from mesh support techniques to identify geometrically
55+
fertile sampling areas more efficiently, reducing brute-force surface traversal.
56+
57+
### Validation
58+
59+
- Integration testing for filters and exclusion zone constraints beyond current unit tests.
60+
- Organic shape testing. Current validation covers box, prism, and small cylinder only.
61+
Complex non-convex geometry is not yet validated.
62+
63+
### Action Server Integration
64+
65+
- Wire `hold_and_weld_gripper_sampler` into the `hold_and_weld_application` action server
66+
system via ROS 2 lifecycle action server. Reference implementations exist in
67+
`hold_and_weld_application`.
68+
69+
### Mesh Support
70+
71+
Mesh based geometry input as an alternative to CAD for sensor driven or CAD unavailable
72+
scenarios. Planned as a two-phase pipeline.
73+
74+
Phase one performs structural analysis of the mesh — identifying graspable regions and
75+
natural surface boundaries through graph based traversal. A spanning tree based approach
76+
is used to guarantee complete mesh coverage and handle topologically circular regions
77+
where no natural boundary exists. CGAL is the targeted library given existing dependency
78+
availability in the system.
79+
80+
Phase two generates ranked contact pairs through guided sampling, starting from
81+
geometrically promising regions and propagating outward via breadth-first search. Surface
82+
boundaries are classified as either geometric features or traversal constructs, kept
83+
strictly separate so ranking reflects only real geometry. Output feeds into the existing
84+
contact pair interface with no downstream changes required.
85+
86+
This is a significant architectural addition planned after core CAD pipeline stabilization.
87+
88+
### Dataset Generation
89+
90+
An output pipeline that takes mesh inputs and produces labeled grasp datasets. The
91+
practical motivation is that weld areas, screw holes, and similar features tend to repeat
92+
across workpieces in the same workplace. A generated dataset allows downstream models to
93+
learn avoidance of these regions without rerunning the full sampling pipeline on every
94+
new workpiece. This would also enable gripping strategies informed by spatial proximity
95+
to weld seams.
96+
97+
### Code Quality
98+
99+
- Add `BRepCheck_Analyzer` validation at geometry entry points for robustness on external
100+
STEP files.
101+
- Fix known error handling inconsistencies per error handling policy. Primarily single
102+
layer catch blocks in `contact_point_sampler.cpp` and unguarded `UIso`/`VIso` handle
103+
checks in `shape_refiner.cpp`.
104+
105+
---
106+
107+
## hold_and_weld_application
108+
109+
### Action Servers
110+
111+
- Worker thread watchdog timeout.
112+
- `detach_object` wiring in `run_job` once re-grasp workflow is defined.
113+
- ACM collision allowance per object instead of per link — required to handle complex
114+
multi-primitive objects correctly where per-link granularity is insufficient.
115+
- `YAML` value validation in `load_job_from_yaml`.
116+
- Online parameter update for execution-time tunable parameters: velocity scaling,
117+
acceleration scaling, controller type.
118+
- Replace `move_to_pose` with proper implementation.
119+
- Limit extra axis movement by parameters.
120+
- Planning scene coordinator to make scene management event driven.
121+
122+
### Coordinator
123+
124+
- Multi-planner pipeline: OMPL for collision-free approach planning, Pilz Industrial
125+
Motion Planner for sharp deterministic path execution — cleaner separation of concerns
126+
than the current single-planner approach.
127+
- Deprecate coordinator in favor of behavior tree integration.
128+
- Scene management via behavior tree nodes.
129+
130+
### Kinematics
131+
132+
- More test coverage on viable and non-viable seam paths.
133+
- Lifecycle harmony with MoveIt 2 — currently requires architectural workarounds due to
134+
MoveIt 2's internal nodes not accepting lifecycle node interfaces.
135+
- Approach configuration finder — derive a guaranteed-valid joint goal from the
136+
validator's IK walk, feed directly to OMPL as goal state, eliminating the current
137+
retry loop entirely.
138+
139+
---
140+
141+
## hold_and_weld_planning
142+
143+
The OCCT pipeline is largely stable. Deterministic seam extraction, normal evaluation,
144+
and pose generation work reliably for the tested joint types. Pipe joint detection
145+
remains incomplete and is not planned for the near term.
146+
147+
The mesh pipeline has one fundamental open problem: CGAL corefinement produces
148+
intersection segments whose endpoints are pinned to triangle edges rather than the true
149+
geometric intersection curve. On simple geometry this approximation is acceptable. On
150+
complex or organic shapes the scatter becomes significant enough to corrupt path
151+
classification and normal extraction downstream.
152+
153+
The candidate approaches to fix this without introducing learned components are:
154+
155+
- **Newton refinement** — project each corefinement vertex onto the true intersection
156+
by iteratively solving the two-tangent-plane system using per-vertex normals from both
157+
meshes. Eliminates tessellation pinning at the cost of one closest-point query per
158+
vertex per iteration.
159+
- **Ridge-valley detection on the edge-contributing mesh** — since all practical weld
160+
joints involve at least one geometric edge, the intersection region always has a
161+
curvature discontinuity on at least one mesh. Detecting that ridge directly may give
162+
a cleaner curve than post-processing corefinement output.
163+
164+
Neither approach is implemented. Both remain as research directions pending
165+
re-engagement with the mesh pipeline.
166+
167+
A new core primitive is planned for complex curves that cannot be adequately represented
168+
as sequences of line and arc segments. This will support point-by-point pose generation
169+
directly from the raw seam geometry, allowing organic and irregular intersection shapes
170+
to be planned without requiring classification into geometric primitives.
171+
172+
A separate planned capability is seam extraction from scanned mesh inputs where CAD
173+
geometry is unavailable. This requires a different pipeline: plane segmentation from the
174+
mesh, intersection computation between segmented planes, and convexity-based validation
175+
to distinguish weld joints from internal mesh edges. This would extend the system to
176+
sensor-driven workflows and real-world scanned geometry where the current CAD-dependent
177+
pipelines cannot be used.
File renamed without changes.

hold_and_weld_gripper_sampler/README.md

Lines changed: 9 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -8,14 +8,6 @@ geometric and environmental requirements. Contact points are computed on actual
88
surfaces giving geometrically exact results. Currently supports CAD-based geometry
99
input only.
1010

11-
## Dependencies
12-
13-
- OpenCASCADE (OCCT) — CAD geometry and topology
14-
- FCL — collision checking
15-
- Embree 4 — watertight ray intersection
16-
- ROS 2 — messaging and lifecycle
17-
- Eigen — linear algebra
18-
1911
## Pipeline
2012

2113
### Geometry Mapper
@@ -49,8 +41,7 @@ clustered pairs.
4941

5042
### Angle Finder
5143

52-
The most technically significant component of the pipeline and the primary location
53-
where approach direction validity is determined. Uses a four phase approach.
44+
Approach direction validity is determined here. Implemented in four phases.
5445

5546
Phase one builds a radial surface map around each contact point by casting
5647
classification rays in decreasing rings starting from finger length radius down to
@@ -90,6 +81,14 @@ sampling regions and acting as collision objects during the Angle Finder phase.
9081
Constraints can represent real objects like ground planes and fixtures or mission
9182
specific forbidden zones like weld seams and screw holes.
9283

84+
## Results
85+
86+
**Grasp candidates example generated by `hold_and_weld_gripper_sampler`:**
87+
88+
![Box Grasp](../docs/cube_grasp.png)
89+
90+
![Cylinder Grasp](../docs/cylinder_grasp.png)
91+
9392
## Quick Start
9493

9594
```bash
@@ -99,8 +98,6 @@ ros2 run hold_and_weld_gripper_sampler grasp_finder_node --config <path_to_yaml>
9998
See `config/grasp_finder_example.yaml` for a fully documented configuration example
10099
covering all supported workpiece types and constraint configurations.
101100

102-
For parameter reference see [PARAMS.md](PARAMS.md).
103-
104101
## Known Limitations
105102

106103
- CAD geometry input only. Mesh and point cloud support is planned.

hold_and_weld_planning/PARAMS.md

Lines changed: 0 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,3 @@
1-
Got it. Here is the Parameters.md:
2-
3-
---
4-
51
# Parameters
62

73
## Workpiece

0 commit comments

Comments
 (0)