🤖 Cooperative Exploration and Mapping Using Dual RGB-D Robots with YOLO-Based Object Detection

---

📌 Project Overview

This repository contains the implementation of a multi-robot autonomous exploration system. The project is based on ROS2 Jazzy and Gazebo and involves two TurtleBot3 Burger robots equipped with LiDAR and RGB-D sensors.
The robots autonomously explore an unknown indoor environment using frontier-based exploration, build a map through SLAM Toolbox, navigate using Nav2, and detect specific objects using a YOLO-based object detection pipeline. Detected objects are filtered, localized in the global map frame, and visualized in RViz2 through a centralized coordination node.

---

🗂 Project Structure

.
├── docker_ws/         # All Docker-related files and scripts to build and manage the development container
├── ros_ws/            # ROS 2 workspace where all project packages are developed
├── run.sh             # Script to start the Docker container
├── exec.sh            # Script to attach a new terminal to the container
├── chown_me.sh        # Fix permissions inside the container
└── README.md

ros_ws/src:

• turtlebot3_gazebo
Adapted from ROBOTIS-GIT/turtlebot3_simulations, includes robot models and launch files to spawn the TurtleBot3 in a Gazebo simulation and enable communication with ROS.

• Added 2 different model files, one for each robot.
• Added 2 different urdf files, one for each robot.
• Updated bridge file duplicating all the bridges by adding robot’s namespace before the topics.

• custom_pkg
Custom package created for the project, which includes nodes, launch and configuration files.

Custom_pkg/
├── config/                           # Configuration files for explore lite, nav2, slam_toolbox
│   ├── explore.yaml
│   ├── nav2_config.yaml
│   └── slam.yaml
├── launch/                           # Launch files for simulation
│   ├── first.launch.py 
│   ├── second.launch.py
│   └──gazebo_multirobot.launch.py
├── rviz/                             # RViz2 configuration file
│   └── rviz_config.rviz
├── resource/                         # Auto-generated by ROS 2
├── test/                             # Default folder for test files (currently unused)
├── custom_pkg/
│   ├── __init__.py                   # Marks the folder as a Python package
│   ├── central_node.py               # Coordination node
│   ├── goal_node.py                  # Goal assignement node
│   ├── map_static_broadcaster.py     # Static transformation node
│   ├── test_node.py                  # Custom node to evaluate the system functionality
│   └── yolo_node.py                  # Object detection node
├── package.xml                       # Package metadata and dependencies
├── setup.cfg                         # Makes package discoverable by ros2 run
└── setup.py                          # Installation script

• custom_msg
Custom package which includes the definition of a message that defines the x,y coordinates of the center of the bounding boxes, the width w, and the height h. It also includes the header frame of the robots from which the message originates.

custom_msg/
├── msg/
│ └── Box.msg       # Definition of the message
├── package.xml 
└── CMakeLists.txt

ros_ws/yolo:

yolo/
├── MR-Project2.v3i.yolov11/   # Personalized YOLO dataset 
├── test_train.py              # Code to test custom YOLO model
├── train_yolo.py              # Code to train custom YOLO model
└── yolo11n.pt                 # Original YOLO model  

---

▶️ First-time setup

1️⃣ Clone Explore Lite

cd ros_ws/src/
git clone https://github.com/robo-friends/m-explore-ros2.git

2️⃣ Build the Docker Image

cd ../../docker_ws/ 
chmod +x build_project.sh 
./build_project.sh

3️⃣ Make Scripts Executable

cd ..
chmod +x run.sh exec.sh chown_me.sh

---

▶️ How to Run the Project

1️⃣ Start the Docker Container

./run.sh

2️⃣ Build and Source the ROS 2 Workspace Inside the container:

colcon build
source install/setup.bash

3️⃣ Launch Simulation and SLAM Run the first launch file

ros2 launch custom_pkg first.launch.py

4️⃣ Launch Navigation and Exploration Open a second terminal, attach it to the same container, and run:

./exec.sh
source install/setup.bash
ros2 launch custom_pkg second.launch.py

---

🧠 YOLO Training and Custom Dataset

Since standard YOLO datasets do not include simple geometric shapes, a custom dataset was created using Roboflow.

To retrain the model:

1. Modify or replace the dataset in ros_ws/src/yolo/
2. Run: python3 train_yolo.py
3. Update the model file used by the yolo_node

---

🔄 Changing World or Robot Configuration

To change the simulation scenario:

• Modify the initial robot positions in first.launch.py and gazebo_multirobot.launch.py
• Change the SDF world file name inside gazebo_multirobot.launch.py
• Choose the correct number of object in the Central Node (line 164/165)

⚠️ The current implementation is designed for exactly two robots. To add more robots, additional URDF/SDF models and corresponding configurations must be created

---

🚀 GPU Acceleration

To enable GPU support for YOLO inference and speed up computation, add the following line to run.sh:

--runtime nvidia --gpus all \

Make sure NVIDIA drivers and NVIDIA Container Toolkit are correctly installed on your system.

If you do not need the 3D viewer, comment-out the gzclient_cmd block in gazebo_multirobot.launch.py.

---

📚 References

• TurtleBot3: https://emanual.robotis.com/docs/en/platform/turtlebot3/overview/
• Nav2: https://docs.nav2.org/
• SLAM Toolbox: https://github.com/SteveMacenski/slam_toolbox
• Explore Lite: https://github.com/robo-friends/m-explore-ros2
• Ultralytics YOLO: https://docs.ultralytics.com/

---

🎥 Simulation Demo

A demonstration video of the autonomous exploration and object detection will be available here:

📽 YouTube Demo

---

📄 Documentation

The full project report, including system architecture, implementation details, and experimental results, is provided as PDF file.

➡️ Download PDF

---

✍️ Credits

This project was developed by Mariapaola Germinario and Sara Laera as part of the Mobile Robotics course at the Politecnico di Bari.