demo_adas

InferenceHelper_Sample_ROS

class_diagram

Node Collection

Node Tested frameworks Description
transport None Just transport input image
cls_mobilenet_v2
  • INFERENCE_HELPER_ENABLE_TFLITE_DELEGATE_XNNPACK
  • INFERENCE_HELPER_ENABLE_ONNX_RUNTIME
  • INFERENCE_HELPER_ENABLE_MNN
  • INFERENCE_HELPER_ENABLE_NCNN
  • INFERENCE_HELPER_ENABLE_OPENCV
Classification by MobileNetV2
det_yolox
  • INFERENCE_HELPER_ENABLE_TFLITE_DELEGATE_XNNPACK
  • INFERENCE_HELPER_ENABLE_ONNX_RUNTIME
  • INFERENCE_HELPER_ENABLE_MNN
  • INFERENCE_HELPER_ENABLE_NCNN
Detection by YOLOX-NanoLight
seg_paddleseg_cityscapessota
  • INFERENCE_HELPER_ENABLE_TFLITE_DELEGATE_XNNPACK
  • INFERENCE_HELPER_ENABLE_ONNX_RUNTIME
Segmentation by PaddleSeg CityScapesSOTAVery heavy
depth_lapdepth
  • INFERENCE_HELPER_ENABLE_TFLITE_DELEGATE_XNNPACK
  • INFERENCE_HELPER_ENABLE_ONNX_RUNTIME
Depth by LapDepth, Point CloudHeavy

Tested Environment

  • Host PC
    • Ubuntu 20.04
    • Ubuntu 20.04 + Docker
    • Windows 11 + WSL2 (Ubuntu 20.04)
  • ROS2 version
    • Foxy

How to Run

Create docker container for ROS2

  • This is just an example. You can skip this if you have your ROS2 environment in you PC

xhost local:    # if needed
docker run -it --name ros2_foxy_1 -e DISPLAY=$DISPLAY --net=host osrf/ros:foxy-desktop
# docker start ros2_foxy_1
# docker exec -it ros2_foxy_1 /ros_entrypoint.sh bash

Install software

# export DISPLAY=192.168.1.2:0   # if needed
apt update && apt install -y libopencv-dev ros-foxy-image-pipeline unzip

Create ROS2 workspace and download this repo

source /opt/ros/foxy/setup.bash
mkdir -p ~/dev_ws/src
cd ~/dev_ws/src

git clone https://github.com/iwatake2222/InferenceHelper_Sample_ROS.git
cd InferenceHelper_Sample_ROS
git submodule update --init
sh inference_helper_sample_ros/src/inference_helper/third_party/download_prebuilt_libraries.sh
sh download_resource.sh

Build

cd ~/dev_ws
rosdep install -i --from-path src --rosdistro foxy -y
colcon build  --cmake-args -DINFERENCE_HELPER_ENABLE_TFLITE_DELEGATE_XNNPACK=OFF -DINFERENCE_HELPER_ENABLE_ONNX_RUNTIME=ON
# colcon build  --cmake-args -DINFERENCE_HELPER_ENABLE_TFLITE_DELEGATE_XNNPACK=ON -DINFERENCE_HELPER_ENABLE_ONNX_RUNTIME=OFF
  • Enable one of INFERENCE_HELPER_ENABLE_*** options when you build.
  • Above commands are just an example. Refer to https://github.com/iwatake2222/InferenceHelper#cmake-options to find more build options
  • Note
    • Some frameworks may not be supported in your environment
    • Some models may not be supported in your environment / framework you selected

Set library path

export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:build/inference_helper_sample_ros/
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:src/InferenceHelper_Sample_ROS/inference_helper_sample_ros/src/inference_helper/third_party/onnxruntime_prebuilt/linux-x64/lib/
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:src/InferenceHelper_Sample_ROS/inference_helper_sample_ros/src/inference_helper/third_party/mnn_prebuilt/ubuntu/lib/
  • You need to specify framework library path in the same terminal as you run the node later
  • There may be a way to avoid it. HELP WANTED!!

Run

  • Run manually
    • You need 3 terminals

cd ~/dev_ws
. install/setup.bash

ros2 run image_publisher image_publisher_node src/InferenceHelper_Sample_ROS/resource/dog.jpg
ros2 run inference_helper_sample_ros det_yolox_exe
ros2 run image_view image_view --ros-args --remap image:=/transported_image_raw
  • Run launch file

cd ~/dev_ws
. install/setup.bash

ros2 launch inference_helper_sample_ros cls_mobilenet_v2.launch.py image_filename:=src/InferenceHelper_Sample_ROS/resource/parrot.jpg
ros2 launch inference_helper_sample_ros det_yolox.launch.py
ros2 launch inference_helper_sample_ros depth_lapdepth.launch.py
ros2 launch inference_helper_sample_ros seg_paddleseg_cityscapessota.launch.py image_filename:=src/InferenceHelper_Sample_ROS/resource/dashcam_02.jpg
  • Run ADAS demo
    • Notice: this demo is very heavy

cd ~/dev_ws
. install/setup.bash

ros2 launch inference_helper_sample_ros adas.launch.py

Node Specs

graph

  • Subscribed Topics
    • /image_raw (sensor_msgs/msg/Image )
  • Published Topics
    • /transported_image_raw (sensor_msgs/msg/Image )
    • /inference_result
      • inference_helper_sample_ros_interface/msg/Classification.msg, BoundingBoxList.msg, etc.
      • sensor_msgs/msg/PointCloud2 (depth)
  • Parameters
    • topic_image_sub
      • /image_raw
    • topic_image_pub
      • /transported_image_raw
    • topic_result_pub
      • /inference_result
    • work_dir
      • /root/dev_ws/src/InferenceHelper_Sample_ROS/resource/
    • thread_num
      • 4

License

  • Copyright 2022 iwatake2222
  • Licensed under the Apache License, Version 2.0

Acknowledgements

  • This project utilizes OSS (Open Source Software)

GitHub

View Github