Creating a Self-Driving Car with ROS2, AI, and Computer Vision

Mar 24, 2023 | Data Science

Welcome to the exciting journey of building a self-driving car using ROS2 (Robot Operating System) combined with deep learning and computer vision. With the help of artificial intelligence, our Prius-like vehicle can recognize traffic signs, track objects, and efficiently follow lanes. Below, we will guide you through the repository setup, features, workflow, and address common troubleshooting issues.

Table of Contents

  1. About This Repository
  2. Using this Repository
  3. Course Workflow
  4. Features
  5. Pre-Course Requirements
  6. Repository Tree
  7. Star History
  8. Link to the Course
  9. Instructors
  10. License

About This Repository

Imagine a car that can autonomously navigate streets just like a Tesla! This repository allows you to create a self-driving car in ROS2 capable of following lanes, classifying sign boards, and performing object tracking to effectively respond to traffic signals and adjust its speed accordingly. You can see this in action by watching our introductory video here.

Using this Repository

To get started with this project, you will need to set up Docker following the comprehensive guides for both Linux and Windows:

If you’re using Ubuntu 20.04, follow the Wiki guide for a seamless setup process.

Course Workflow

  • ROS Package
    • World Models Creation
    • Prius OSRF Gazebo Model Editing
    • Nodes, Launch Files
    • SDF through Gazebo
    • Textures and Plugins in SDF
  • Computer Vision
    • Perception Pipeline setup
    • Lane Detection with Computer Vision Techniques
    • Traffic Light Detection Using Haar Cascades
    • Sign and Traffic Light Tracking using Optical Flow
    • Rule-Based Control Algorithms
  • Deep Learning
    • Sign Classification using a custom-built CNN

Features

Our self-driving car comes packed with astonishing features:

  • Prius Hybrid Car: ![Prius Hybrid Car](https://github.com/noshluk2/ROS2-Self-Driving-Car-AI-using-OpenCV/blob/main/Images/videos/the_car.gif)
  • Satellite Navigation (NEW!):
    • Stage 1: Localization ![Localization](https://github.com/noshluk2/ROS2-Self-Driving-Car-AI-using-OpenCV/blob/main/Images/videos/Sat_Nav1_localization.gif)
    • Stage 2: Mapping ![Mapping](https://github.com/noshluk2/ROS2-Self-Driving-Car-AI-using-OpenCV/blob/main/Images/videos/Sat_Nav2_mapping.gif)
    • Stage 3: Path-Planning ![Path Planning](https://github.com/noshluk2/ROS2-Self-Driving-Car-AI-using-OpenCV/blob/main/Images/videos/Sat_Nav3_pathplanning.gif)
    • Stage 4: Motion-Planning ![Motion Planning](https://github.com/noshluk2/ROS2-Self-Driving-Car-AI-using-OpenCV/blob/main/Images/videos/Sat_Nav4_motionplanning.gif)
  • Lane Following: ![Lane Following](https://github.com/noshluk2/ROS2-Self-Driving-Car-AI-using-OpenCV/blob/main/Images/videos/lane_detection.gif)
  • Sign Board Detection: ![Sign Board Detection](https://github.com/noshluk2/ROS2-Self-Driving-Car-AI-using-OpenCV/blob/main/Images/videos/traffic_signs_boards.gif)
  • Traffic Signal Recognition: ![Traffic Signal Recognition](https://github.com/noshluk2/ROS2-Self-Driving-Car-AI-using-OpenCV/blob/main/Images/videos/traffic_signal.gif)
  • T-Junction Navigation: ![T-Junction Navigation](https://github.com/noshluk2/ROS2-Self-Driving-Car-AI-using-OpenCV/blob/main/Images/videos/j_turning.gif)
  • The World: ![The World](https://github.com/noshluk2/ROS2-Self-Driving-Car-AI-using-OpenCV/blob/main/Images/videos/world.gif)
  • Custom Models: ![Custom Models](https://github.com/noshluk2/ROS2-Self-Driving-Car-AI-using-OpenCV/blob/main/Images/videos/custom_models.gif)

Pre-Course Requirements

Before diving in, ensure you have the following:

  • Software Based:
    • Ubuntu 20.04 (LTS)
    • ROS2 – Foxy Fitzroy
    • Python 3.6
    • OpenCV 4.2
    • TensorFlow 2.14
  • Skill Based:
    • Basic ROS2 Nodes Communication
    • Launch Files
    • Gazebo Model Creation
    • Basic OpenCV Usage
    • Motivated Mind 🙂

Repository Tree

Understanding the structure of the repository is crucial. The tree diagram illustrates important files and their functions:

![Repository Structure](https://github.com/noshluk2/ROS2-Self-Driving-Car-AI-using-OpenCV/blob/main/Images/videos/ROS2_SDC_Tree.png)

Star History

Star History Chart

Ready to take your skills to the next level? Enroll now with a special discount on this course: Discounted Link.

Instructors

Learn from the best:

  • Haider Najeeb (Computer Vision) – Profile Link
  • Muhammad Luqman (ROS Simulation and Control Systems) – Profile Link

License

This project is distributed under the GNU-GPL License. For more information, please refer to the LICENSE file.

Troubleshooting

If you encounter issues during setup or usage, consider the following tips:

  • Ensure all software prerequisites are met, particularly the correct version of Ubuntu and packages.
  • Double-check Docker installation commands for any discrepancies.
  • Refer to the Wiki or GitHub Issues page for any reported issues and resolutions.
  • Make sure you follow the course workflow step by step, as skipping instructions may cause unexpected behavior.

If you still face problems, join our community or cite us at **[fxis.ai](https://fxis.ai)** for assistance or project collaboration suggestions.

At **[fxis.ai](https://fxis.ai)**, we believe that such advancements are crucial for the future of AI, as they enable more comprehensive and effective solutions. Our team is continually exploring new methodologies to push the envelope in artificial intelligence, ensuring that our clients benefit from the latest technological innovations.

Stay Informed with the Newest F(x) Insights and Blogs

Tech News and Blog Highlights, Straight to Your Inbox

Let’s Build Success Together