Welcome to the world of real-time multi-object recognition! In this guide, we will explore how to use the YOLOv8n model with Unity Sentis to enable object detection capabilities in Unity 2023. Version 1.4.0-pre.3 is the latest iteration of this integration, and if you’re ready to dive in, let’s get started!

Getting Started

Before we jump into the implementation, make sure you have the following components in place:

• Unity 2023 installed on your system.
• Access to the Unity Package Manager.
• The YOLOv8n model, which can be downloaded from this link.

How to Use

Follow the steps below to set up YOLOv8n with Unity Sentis:

1. Install the Sentis Package: Open the Unity Package Manager and get the package com.unity.sentis.
2. Create a New Scene: In Unity, start by creating a new scene.
3. Install the Latest Sentis Version: Make sure to install com.unity.sentis version 1.4.0-pre.3 from the package manager.
4. Add the C# Script: Attach the provided C# script to the Main Camera in your scene.
5. Set Up the Display: Create a Raw Image in the scene and link it to the displayImage in your script.
6. Model and Labels: Drag the yolov8n.sentis file into the model asset field and drag classes.txt onto the labelAssets field.
7. Video Setup: Place your video file in the Assets/StreamingAssets folder. Set the videoName to match the filename in your script.
8. Bounding Box Setup: Set the fields for the bounding box texture sprite (you can create your own one using a transparent texture or utilize an inbuilt one) and select the desired font.

Visual Confirmation

If everything is set up correctly, you should see a real-time view similar to the one below:

![preview](preview.png)

Troubleshooting Tips

In case you encounter any issues with your setup, here are some troubleshooting ideas:

• Ensure that you are using Unity 2023, as older versions may not be compatible.
• Verify that the com.unity.sentis package is correctly installed and at the required version (1.4.0-pre.3).
• Make certain that all file paths (e.g., video files and assets) are correct and properly linked in the script.
• If you experience display issues, double-check your Raw Image settings and the linked script components.

For more insights, updates, or to collaborate on AI development projects, stay connected with fxis.ai.

Additional Information

Keep in mind that the Non-Maximum Suppression (NMS) selection feature will see improvements in future releases of Sentis. Currently, the model employs a singular-class approach for object detection. For additional details on Unity Sentis, visit this page.

Licensing Information

The YOLO models are distributed under the GPLv3 license, ensuring that these powerful tools are available for collaborative use within the open-source community.

At fxis.ai, we believe that such advancements are crucial for the future of AI, as they enable more comprehensive and effective solutions. Our team is continually exploring new methodologies to push the envelope in artificial intelligence, ensuring that our clients benefit from the latest technological innovations.