ofxFaceTracker3

Working in progress

Fast and robust face tracking addon for openFrameworks based on YOLO5Face and ONNX Runtime.

Features

  • Fast and robust face & keypoints detection using YOLO5Face.
  • Achieve realtime FPS on both CPU and GPU.

Tested environment

  • oF0.11.2 + macOS Catalina Intel CPU
  • oF0.11.2 + Windows10 CPU / CUDA / TensorRT
    • CUDA 11.4, TensorRT 8.0.3.4

Installation

  • This addon depends on following addons. Please pull them to ${OF_BASE_PATH}/addons directory first.
  • Generate project using project generator, then model directory is copied into bin/data.

Usage

  • In model directory, there are 2 converted pretrained models, which are yolov5s-face_640x640.onnx and yolov5n-face0.5_320x320.onnx.
    • yolov5s-face_640x640.onnx is suitable for GPU detection, and yolov5n-face0.5_320x320.onnx is suitable for CPU detection with slightly accuracy degradation.
    • Original PyTorch pretrained models can be downloaded from here.
    • Then onnx files are generated using this script.
  • ofxFaceTracker3::Tracker::setupCpu(); is handy setup method for CPU detection, which loads yolov5n-face0.5_320x320.onnx by default.
  • ofxFaceTracker3::Tracker::setupGpu(); is handy setup method for GPU detection, which loads yolov5s-face_640x640.onnx by default.
  • See example-facetracker3 for more details.

Notes

  • If TensorRT is enabled, it takes long time when starting app for the first time. In my environment, it takes 12 minutes. Then converted *.trt file is generated under bin/data/model/yolov5s-face_640x640_trt_cache directory.

Comparison to ofxFaceTracker2

  • For easy environment such as no difficult lighting, no occulusion and frontal angle, ofxFaceTracker2 might be better because it runs on CPU with super lightweight load.
  • However ofxFaceTracker2 hardly detects masked faces which are common in COVID-19 era, and also it does not support non-frontal faces.
  • If you guys face any of difficult detection conditions, ofxFaceTracker3 will perform lots better than ofxFaceTracker2.

Reference

GitHub

View Github