Biped-PyMPC

Official codebase for "RL-augmented Adaptive Model Predictive Control for Bipedal Locomotion over Challenging Terrain"

This project presents a batched MPC controller designed for learning and model-based hybrid combined approach. We open-source a batched QP solver implemented in CusADi, along with kinematics and dynamics code written purely in PyTorch.

Note

We currently only support bipedal robot HECTOR. Although we have code for Booster T1 robot, walking controller is not tested yet.

Installation

pip install -e .

Code generation

If you are just interested in running, code see download precompiled files. If you want to write your own and compile it, see generate code in local.

download precompiled files

You can pull files from google using gdown.

./script/setup.sh

generate code in local

Generate CasADi function

# generate functions that form QP matrices and solve this QP problem.
python3 -m biped_pympc.casadi.srbd_constraints
python3 -m biped_pympc.casadi.generate_solver_function

This generate casadi function file ready to be compiled as cuda kernel.

Generate CusADi function

To run CusADi, you need to compile your casadi code as .so file.
We assume your base casadi function files are saved in biped_pympc/cusadi/src/casadi_functions.

# compile casadi as cusadi .so file

# compile dynamics code
python3 cusadi/run_codegen.py --fn qp_former
cp cusadi/build/qp_former.so cusadi/src/cusadi_functions/qp_former.so

# compile solver code
python3 cusadi/run_codegen.py --fn sparse_pdipm_multiple_iterations
cp cusadi/build/sparse_pdipm_multiple_iterations.so cusadi/src/cusadi_functions/sparse_pdipm_multiple_iterations.so

IsaacLab example

We use this batched MPC controller in IsaacLab to train our RL-augmented MPC policy. For full documentation, please refer to IsaacLab repository: https://github.com/rl-augmented-mpc/IsaacLab4.5/tree/devel/scripts/biped

Note

We only provides preliminary results on pyramid stairs. Furthermore, the batched MPC controller is currently not as performative as C++ MPC controller. The future work involves performance improvement and reproducing results on multiple terrains as done using C++ controller.

WIP List

Initial code base implementation

• Implement biped.py
• Implement robot_data.py
• Implement gait_generator.py
• Implement state_estimator.py
• Implement swing_leg_trajectory.py
• Implement qp solver
• Functional SRBD MPC
• Implement swing_leg_controller.py
• Implement leg_controller.py
• Implement bipedal_controller.py (equivalent of FSMWalking.cpp)
• Check code in Genesis
• Check code in Isaac

Interface and functionalities

• Add IsaacLab example
• Add RL-MPC interface
• Add CusADi solver
• Create ROS2 wrapper

Update

• Create QP solver in warp sparse class.

Limitation

The current codebase suffers from long compilation times because CusADi compiles the generated SX instructions one by one. This forced us to limit full Newton iterations to a maximum of 5 within a single QP solve. Even so, compiling 5 Newton iterations takes around 3 hours, which is a significant bottleneck. Future work involves replacing the CusADi-based solver with one written purely in cuSparse, CuDSS or Warp.

Acknowledgement

We appreciate Se Hwan Jeon for his support in setting up CusADi and open-sourcing the code.