In the realm of autonomous driving, detecting and tracking vehicles is a crucial capability that paves the way for safer navigation. In this blog, we’ll unravel the process of utilizing a camera mounted inside a self-driving car for vehicle detection and tracking. Our aim is to provide a simple framework for developers, researchers, and engineers to experiment with different detectors and tracking algorithms.

Overview of the Process

The vehicle detection and tracking process can be likened to a traffic officer who stands at an intersection, observing the flow of vehicles. Initially, the officer identifies each vehicle (detection), and then tracks their movements as they navigate through the intersection (tracking). In our implementation, we follow a systematic approach:

• Initialize the detector and tracker.
• Localize the vehicles in each video frame using the detector.
• Update the tracker with the detection results.
• Annotate and display the tracking results in the video frame.

Key Files in This Repository

• detector.py: Contains the CarDetector class for vehicle detection.
• tracker.py: Implements the Kalman Filter for tracking predictions and updates.
• main.py: Manages the detection-tracking pipeline, including assignment and tracking management.
• helpers.py: Includes a collection of helpful functions.
• ssd_mobilenet_v1_coco_11_06_2017frozen_inference_graph.pb: Pre-trained MobileNet-COCO model.

Step 1: Detection of Vehicles

The vehicle detection phase involves processing captured images and generating bounding boxes around identified vehicles. We utilize the TensorFlow Object Detection API, which is an open-source framework designed for this purpose. In this case, the lightweight model, ssd_mobilenet_v1_coco, strikes a balance between accuracy and performance. It identifies vehicles, including cars, buses, and trucks, based on their class IDs.

Step 2: Implementing Detection

In our vehicle detection implementation, we capture bounding box coordinates for the detected vehicles. The code snippet below illustrates how detections are obtained using TensorFlow API:

(boxes, scores, classes, num_detections) = self.sess.run(                  
                  [self.boxes, self.scores, self.classes, self.num_detections],                  
                  feed_dict={self.image_tensor: image_expanded})   

Here, boxes correspond to bounding boxes, scores represent confidence levels, and classes align with vehicle types. To refine our selection, we filter out detections with low confidence, specifically those with a score greater than 0.3. This process is akin to a bouncer at a club, only allowing guests with a certain level of confidence to enter.

Step 3: Kalman Filter for Tracking

To track vehicles effectively, we employ a Kalman Filter, a powerful mathematical tool that predicts an object’s future location, corrects past predictions based on new measurements, and reduces noise from inaccurate detections. Imagine a navigator that not only charts a course but also adjusts the route based on real-time information.

The Kalman filter operates in two phases – Prediction and Update:

• Prediction: Uses previous states to forecast current states.
• Update: Utilizes current measurements (bounding box location) to refine predictions.

Step 4: Assignment of Detections to Trackers

In scenarios where multiple vehicles are detected, we need to assign tracks accurately. This is done using the intersection over union (IOU) metric. Think of this process like a matchmaker at a party, ensuring that each guest finds their partner based on shared characteristics.

Step 5: Creating the Pipeline

The tracking pipeline includes parameters that help manage how detections are matched to trackers:

• min_hits: Minimum consecutive matches needed to establish a track.
• max_age: Maximum number of unmatched detections before a track is deleted.

The pipeline processes matches and keeps track of unmatched detections using conditional statements, akin to keeping score during a game. This organization ensures that our tracking remains robust and efficient.

Troubleshooting Common Issues

While our detection and tracking system functions well, one common issue arises during occlusions. This occurs when one vehicle obstructs another, causing confusion in detection. Just as a traffic officer might misidentify a single large vehicle instead of two closely passing cars, our system can struggle to maintain accurate tracking in such scenarios.

To mitigate occlusions and refine tracking performance:

• Experiment with different detection thresholds.
• Adjust parameters in the Kalman filter.
• Implement additional logic to handle close proximity scenarios.

For more insights, updates, or to collaborate on AI development projects, stay connected with fxis.ai.

Conclusion

At fxis.ai, we believe that such advancements are crucial for the future of AI, as they enable more comprehensive and effective solutions. Our team is continually exploring new methodologies to push the envelope in artificial intelligence, ensuring that our clients benefit from the latest technological innovations.