Welcome to the world of AI evaluation! If you’ve trained a computer vision model and want to assess its performance with minimal hassle, you’re in the right place! In this guide, we will walk you through the steps of setting up the BMW AI Evaluation GUI, a powerful tool designed for model evaluation in both object detection and image classification.

Prerequisites

Before diving into the setup, ensure you have the following prerequisites:

• Ubuntu 18.04 or higher
• Docker CE (latest stable release)
• Docker-Compose

Checking for Prerequisites

To verify if you have the required software installed, run the following commands in your terminal:

• Check Docker: docker --version
• Check Docker Compose: docker-compose --version

Installing Prerequisites

If Docker or Docker Compose is not installed, follow these steps:

1. For both Docker and Docker Compose installation:

chmod +x install_full.sh
source install_full.sh

2. If you already have Docker but need Docker Compose:

chmod +x install_compose.sh
source install_compose.sh

Making Changes for Configuration

Your next task involves configuring the settings for the evaluation GUI:

• Navigate to guisrc/environments/environment.ts and guisrc/environments/environment.prod.ts and update the field url to match your machine’s IP address. Use the ifconfig command to find your IP address (it should start with 10., 172.16., or 192.168.).
• If you’re behind a proxy, enter your proxy settings in the base-dir/proxy.json file.

Dataset Folder Structure

Your dataset must be organized according to specific structures. Here’s how you should structure your datasets:

For Object Detection:

datasets
├── object_detection
│   └── sample_dataset
│       ├── images
│       │   ├── img_1.jpg
│       │   └── img_2.png
│       └── labels
│           ├── json
│           │   ├── img_1.jpg
│           │   └── img_2.png

For Image Classification:

datasets
├── image_classification
│   └── classification_dataset
│       ├── cat
│       │   ├── img_1.jpg
│       │   └── img_2.png
│       └── dog
│           ├── img_1.jpg
│           └── img_2.png

Building and Running the Solution

Once your environment is set and datasets are prepared, follow these commands to build and run the solution:

1. From the repository’s root directory, build the solution:

docker-compose -f build.yml build

2. Run the solution:

docker-compose -f run.yml up

Using the Evaluation GUI

After a successful setup, access the evaluation GUI:

• For local access, type localhost:4200 or 127.0.0.1:4200 in your web browser.
• If hosted remotely, use machine_ip:4200.

Steps to Use the Evaluation GUI:

1. Add an Inference Service

To start any job, you need to add an inference service. Avoid using http://localhost:port or http://127.0.0.1:port.

2. Start a Job

Provide the dataset folder or upload a zip file, choose your inference service, and select the appropriate evaluation metric, like **IoU grouping** for object detection.

3. Browse Results

Batch results are visible during training and detailed results can be downloaded post-job completion.

4. Output Results

Access output results via a zip file or through the specified output folder. This includes general evaluations and per-label evaluations.

5. Remove/Stop a Job

Don’t forget to stop a job when you’re done!

Troubleshooting

If you encounter issues during installation or usage, consider these common troubleshooting tips:

• Ensure Docker and Docker Compose are correctly installed and running.
• Double-check your dataset folder structure for errors.
• Verify your proxy settings if applicable.

For more insights, updates, or to collaborate on AI development projects, stay connected with fxis.ai.

Conclusion

Once you’ve finished running evaluations and analyzing output results, you’ll have a clearer understanding of how your computer vision model performs. The BMW AI Evaluation GUI makes it easy to visualize and interpret evaluation metrics, enabling you to take actionable steps based on your findings.

At fxis.ai, we believe that such advancements are crucial for the future of AI, as they enable more comprehensive and effective solutions. Our team is continually exploring new methodologies to push the envelope in artificial intelligence, ensuring that our clients benefit from the latest technological innovations.