mediapipe_track

• ROS2 package that applies MediaPipe Pose solution
• Important addition to MediaPipe Pose: ability to calculate and detect the person's 3D position and publish this information in ROS Topics so the robot can know its relative position from the person detected
• Synchronous processing in real-time frames can be enabled/disabled by a service
• Assynchronous processing responds to Action requests
• Tested in Ubuntu 22 with ROS Humble

Installation

Building

cd <ros2_ws>/src
git clone https://github.com/UtBotsAtHome-UTFPR/mediapipe_track.git
cd ..
colcon build --packages-select mediapipe_track utbots_actions utbots_srvs utbots_msgs \
--allow-overriding utbots_msgs utbots_actions utbots_srvs \
&& source install/setup.bash

Dependencies

To avoid conflicts between package dependencies, we use virtual environments. Change the virtuelenv path in the executable field in setup.cfg. Not the ideal solution, but the current one while we don't use Docker.

If you haven't installed virtualenv:

pip3 install virtualenv

Create and activate env:

python -m virtualenv <env_path>
source <env_path>/bin/activate

The code runs on Python 3.9 and depends on mediapipe 0.10.21 for the latest features. Install the requirements:

pip install -r requirements.txt

Running

To run the Mediapipe pose estimation node:

ros2 run mediapipe_track mediapipe_node

Or using launchfiles:

ros2 launch mediapipe_track mediapipe_node.launch.py

With launchfiles you can specify the parameter values using any of the arguments in Command Line or other launchfiles (for instance, disabling draw_skeleton_img or calculate_torso_point could save processing usage):

'topic_namespace':
    Namespace for topics
    (default: '/utbots/vision')

'model_path':
    Path to the model file
    (default: '')

'num_poses':
    Maximum number of poses to detect
    (default: '1')

'detection_conf':
    Detection confidence threshold
    (default: '0.75')

'presence_conf':
    Presence confidence threshold
    (default: '0.5')

'track_conf':
    Tracking confidence threshold
    (default: '0.9')

'draw_skeleton_img':
    Draw skeleton image
    (default: 'true')

'calculate_torso_point':
    Calculate torso point
    (default: 'true')

'segmentation_mask':
    Use segmentation mask
    (default: 'false')

'rgb_topic':
    RGB input topic
    (default: '/camera/rgb/image_color')