How to Implement YoloV8 with TensorRT in C++

Welcome to the world of object detection! Today, we will dive into using YoloV8, a powerful visual recognition model, alongside TensorRT in C++. This guide will walk you through setting up your environment, converting models, building the project, and running inference tasks. Ready? Let’s start!

Getting Started

This project allows you to leverage the TensorRT C++ API for GPU inference using YoloV8. Before we get into the technical details, ensure that you’re familiar with the TensorRT C++ API project as it serves as the backbone for running inferences.

Prerequisites

Before we begin, make sure you have the following in place:

• Ubuntu 20.04 or 22.04 (Windows is not supported at this time).
• CUDA installation. (Recommended version: 12.0). You can find installation instructions here.
• cuDNN (Recommended version: 8). Instructions can be found here.
• Run: sudo apt install build-essential
• Python pip for CMake: sudo apt install python3-pip followed by pip3 install cmake
• OpenCV with CUDA support. For compilation instructions, follow the build_opencv.sh script.
• TensorRT 10. Download it here.

Installation Steps

1. Clone the repository with recursive submodules:

   git clone https://github.com/cyrusbehr/YOLOv8-TensorRT-CPP --recursive

Model Conversion from PyTorch to ONNX

To use YoloV8, convert your PyTorch model to ONNX:

• Get the desired model version from the official YoloV8 repository.
• Install ultralytics library with: pip3 install ultralytics
• Navigate to your scripts directory and run:

  python3 pytorch2onnx.py --pt_path path_to_your_pt_file

Building the Project

1. Navigate to your project folder and create a build directory:

   mkdir build && cd build && cmake .. && make -j

Running Executables

With the setup done, it’s time to run your scripts. The first run might take some time (~5 mins) as TensorRT optimizes the engine file.

• Benchmarking script:

  ./benchmark --model path_to_your_onnx_model.onnx --input path_to_your_benchmark_image.png

• Object detection on image:

  ./detect_object_image --model path_to_your_onnx_model.onnx --input path_to_your_image.jpg

• Real-time webcam inference:

  ./detect_object_video --model path_to_your_onnx_model.onnx --input 0

• For a full list of arguments, just run the executables without providing any parameters.

INT8 Inference

For enhanced speed using INT8 precision, ensure to supply calibration data representative of your inference data. Use 1K+ calibration images.

• Example calibration command:

  --precision INT8 --calibration-data path_to_your_calibration_data

• If you encounter an out of memory error, decrease the Options.calibrationBatchSize value.

Benchmarking

To ensure efficient performance, verify that your GPU is unloaded before benchmarking.

Troubleshooting

If issues arise during the TensorRT engine file creation from the ONNX model, modify the log level in lib/tensorrt-cpp-api/src/engine.cpp to kVERBOSE for more detailed diagnostics. Rebuild and run the program again to glean insights into the failure point.

For more insights, updates, or to collaborate on AI development projects, stay connected with fxis.ai.

Conclusion

At fxis.ai, we believe that such advancements are crucial for the future of AI, as they enable more comprehensive and effective solutions. Our team is continually exploring new methodologies to push the envelope in artificial intelligence, ensuring that our clients benefit from the latest technological innovations.