Human-Robot Gym

Human-Robot Gym is a training environment for safe human-robot collaboration. This project is designed for training autonomous agents in human-robot collaboration tasks. We provide environments, human animations, and the human safety functionality. We further provide optional training scripts for training RL agents using stable-baselines3.

Documentation

Read the documentation

Installation

Clone the repo with submodules

git clone --recurse-submodules git@github.com:TUMcps/human-robot-gym.git

[Optional] Setup anaconda environment

Install anaconda. Add conda-forge to your channels with

conda config --add channels conda-forge
conda config --set channel_priority strict

and create the hrgym conda environment:

conda env create -f environment.yml
conda activate hrgym

You can also use your preferred python environment.

Install the failsafe controller / safety shield

The installation requires gcc, c++>=17, and Eigen3 version 3.4 (download it here: https://eigen.tuxfamily.org/index.php?title=Main_Page). Set the path to your eigen3 installation to this env variable, e.g.,

export EIGEN3_INCLUDE_DIR="/usr/include/eigen3/eigen-3.4.0"

Now run

cd human-robot-gym/human_robot_gym/controllers/failsafe_controller/sara-shield
git submodule init
git submodule update --recursive
pip install -r requirements.txt
pip install .

Install the human-robot-gym

cd human-robot-gym
pip install .

Optional:

• Install with development flags

pip install -e .[dev]

• Install stable baselines 3 to directly use our train scripts:

pip install .[training]

Post Install: Run the setup macros script that robosuite asks you to build. E.g.:

python /opt/conda/envs/hrgym/lib/python3.13/site-packages/robosuite/scripts/setup_macros.py

Test a demo

python human_robot_gym/demos/demo_gym_functionality_panda.py

You can find more useful demos in the human_robot_gym/demos folder.

Run a RL training

python human_robot_gym/training/train_SB3.py -cn human_reach_ppo_parallel

You can activate weights and biases with

python human_robot_gym/training/train_SB3.py -cn human_reach_ppo_parallel run.type=wandb

Known issues

SARA shield installation

INCLUDE_DIRECTORIES given empty-string as include directory

During installation of SARA shield:

CMake Error at CMakeLists.txt:29 (INCLUDE_DIRECTORIES):
INCLUDE_DIRECTORIES given empty-string as include directory.

is caused by a missing Eigen installation or incorrect EIGEN3_INCLUDE_DIR.

GTest missing

During installation of SARA shield:

Could NOT find GTest (missing: GTEST_LIBRARY GTEST_INCLUDE_DIR
        GTEST_MAIN_LIBRARY)

Solution 1 Make sure that you have gtest installed with

sudo apt-get install libgtest-dev

Solution 2 Disable testing in SARA shield build

BUILD_TESTS=OFF pip install .

Runtime issues

`GLIBCXX_3.4.29' not found

ImportError: /lib/x86_64-linux-gnu/libstdc++.so.6: version `GLIBCXX_3.4.29' not found (required by /opt/conda/envs/hrgym/lib/python3.8/site-packages/google/protobuf/pyext/_message.cpython-38-x86_64-linux-gnu.so

Solution

sudo apt-get install software-properties-common
sudo add-apt-repository ppa:ubuntu-toolchain-r/test
sudo apt-get update
sudo apt-get upgrade libstdc++6
sudo apt-get dist-upgrade

libGLEW.so from LD_PRELOAD cannot be preloaded

The error

ERROR: ld.so: object '/usr/lib/x86_64-linux-gnu/libGLEW.so' from LD_PRELOAD cannot be preloaded (cannot open shared object file): ignored.

can be fixed by adding

export LD_PRELOAD=/usr/lib/x86_64-linux-gnu/libGLEW.so

to your bashrc.

Developer's guide

To check your code for linting style:

flake8
pydocstyle

Recommended tool for auto-formatting:

pip install black

Dockerization

There are two Dockerfiles

• Dockerfile.build is the minimum environment neccessary to build and run human-robot-gym. This image is used by our CI pipeline.
• Dockerfile.dev is based on Nvidia cuda and is therefore capable of running trainings on the GPU and also provide a visual output as if you would run the code locally.

Building the image

We automated the creation of the Dockerfile.dev docker image. Simply run

./build_docker.sh root

Running an iterative container

After creating the image with ./build_docker.sh, run

./run_docker.sh root gui

to create an iterative container with GUI support, or run

./run_docker.sh root 

if you don't need the GUI (i.e., training only). Our entrypoint.sh builds safety_shield_py and human_robot_gym when starting up the container. We defined the CMD as /bin/bash, so that you can use the interactive console after creation. Now, enjoy your docker container.

Build the Documentation

To generate the documentation locally, install

apt install sphinx graphviz
pip install sphinx_rtd_theme
pip install sphinx-autoapi
pip install graphviz

Then make the documentation with

cd docs
make html

Inspect the documentation at docs/build/html/index.html.

ICRA 2024 Experiment Reproduction

We updated our benchmark environments! This new version supports many requested updates:

• python: 3.8 -> 3.13
• robosuite: 1.3.1 -> 1.5
• mujoco: mujoco_py -> mujoco
• gym: gym -> gymnasium
• stable-baselines3: 1.5 -> 2.7 These updates will likely impact the training performance. If you would like to reproduce the exact ICRA 2024 results, please check out the icra2024 branch.

Not tested, proceed with caution

To reproduce the results described in the human-robot-gym paper, execute

./icra_2024_run_experiments_in_docker.sh

This will

• Create a docker image,
• Run a container, and
• Run all experiments within the container.

Please note that due to the amount of training runs (6 environments x 4 methods x 5 random seeds), the execution of this command will take a very significant amount of time and will run on up to 50 threads.