Gesture-Controlled-Omnidirectional-Robot

This is the official repository for the omni directional hand gesture robot project at IvLabs, VNIT Nagpur.

Objectives

The objectives of the summer internship were to create a omni directional robot which move in all 8 direction using transmitter receiver with hand gesture.

Methodology

Software Implementation:

1. Connection of MPU6050 with ESP32

Connected an MPU6050 to an ESP32 as the following connection. The MPU6050 is an inertial measurement unit(IMU) that is a combination of gyroscope and accelerometer in all 6 dimension, which can detect the orientation, motion, and acceleration of the hand and give its value of each change to ESP32.

2. Simulation result of MPU6050

MPU6050 track each movement and send its value to ESP32 as shown in the result shown below.

3. Motion of Omnidirectional robot

(i) Forward motion      (ii) Backward motion      (iii) Leftward motion      (iv) Rightward motion

(v) Anti-Clockwise Rotation    (vi) Clockwise Rotation

(vii) Forward-right motion    (viii) Backward-right motion    (ix) Forward-left motion    (x) Backward-left motion

Motion of robot with wheel rotating direction

4. Speed Control with PWM Pins

• Purpose : To control the speed of the motors driving the Mecanum wheels, we employed Pulse Width Modulation (PWM). This technique allows us to simulate varying levels of voltage by rapidly switching a digital signal between high (on) and low (off) states.

• How PWM Pins Works :

  • The PWM signals are sent to the L298N motor driver through its enable pins (ENA and ENB). By adjusting these signals, we can finely control the speed of each motor based on user gestures.

• Procedure :

  • Used the ESP32 PWM pins to generate Pulse Width Modulation (PWM) signals for the L298N motor driver.
  • Maped gesture intensities to speed levels (e.g., tilt angle to PWM duty cycle).
  • Ensure smooth acceleration and deceleration by fine-tuning the PWM signals.

• Outcome : Enables precise control of motor speeds, ensuring the robot moves efficiently and responsively.

• 5. Wireless Data Transfer using WiFi (ESP-NOW Protocol)

• Procedure:

  • Used ESP32 modules for both transmitting and receiving data.
  • Implemented the ESP-NOW protocol, which allows peer-to-peer communication with low latency.
  • Transmited gesture data from the glove as control commands (e.g., direction, speed).

Circuit of Hand-Gesture IMU :

Circuit of Omni-Directional Robot:

Final Outcome

1] Forward Motion and Gesture

2] Backward Motion and Gesture

3] Leftward motion and Gesture

4] Rightward motion and Gesture

5] Backward-Right motion and Gesture

6] Forward-Right motion and Gesture

7] Backward-Left motion and Gesture

8] Forward-Left motion and Gesture

9] Clockwise motion and Gesture

10] Anti-Clockwise motion and Gesture

Link to extensive breakdown of project: https://hackmd.io/@r-D0tnDWS9yZpWf3XtGu4g/S1byF9wJJx