How to Create a Self-Driving Mario Kart with TensorFlow

Oct 2, 2020 | Data Science

Ever dreamed of racing your favorite Mario Kart character without actually holding the controller? With TensorKart and TensorFlow, you can turn that dream into reality! In this blog, we’ll walk you through setting up a self-driving Mario Kart model that can navigate various race tracks. Let’s dive into the intricacies of this exciting project!

Prerequisites: Setting Up Your Environment

Before you start, ensure that your development environment is ready. Here’s what you need:

  • Python installed on your system.
  • Install dependencies by running:
  • pip install -r requirements.txt
  • Install the Mupen64Plus emulator using:
  • apt-get install mupen64plus

Recording Samples: Capturing Data

To train your model, you need to first gather some data. Here’s how you can record samples:

  1. Start your Mupen64Plus emulator and run Mario Kart 64.
  2. Ensure that your joystick is connected and configured correctly with the SDL input plugin.
  3. Run the recording script:
  4. python record.py
  5. Check if the graph responds to your joystick input.
  6. Position the emulator window in the top left corner to capture images, then press “record” and play through a race level. You can later edit your data capture by trimming images from the data.csv file.

Make sure to keep an eye on the samples you recorded, as sometimes the desktop might end up in the screenshots!

Preparing Your Training Data

Once your samples are recorded, you need to prepare the training data:

python utils.py prepare samples*

This command will generate your input (X) and output (y) matrices for training:

  • X contains a 3-dimensional array of images.
  • y represents the expected joystick output:
    • [0] – Joystick X-Axis
    • [1] – Joystick Y-Axis
    • [2] – Button A
    • [3] – Button B
    • [4] – Button RB

Training Your Model

With your data prepared, you’re now ready to train your model:

python train.py

Keep in mind that training can take around an hour, depending on your system specifications and the amount of data you’re working with. The best model will be saved after all training epochs are completed.

Playing: Racing with Your AI

Now that your model is trained, it’s time to see it in action!

python play.py

This command will utilize the gym-mupen64plus environment to execute the AI against the Mario Kart tracks, providing an interactive adventure like no other!

Troubleshooting: Common Issues

If you encounter issues or your model doesn’t behave as expected, consider the following troubleshooting tips:

  • Ensure your joystick is fully operational and properly configured with Mupen64Plus.
  • If recordings are unresponsive or show desktop images, double-check the emulator positioning and adjust as necessary.
  • Make sure you are capturing input properly by testing your joystick functionality.

For more insights, updates, or to collaborate on AI development projects, stay connected with fxis.ai.

Future Work Ideas

Looking forward, there are many exciting directions this project could take:

  • Integrate a reinforcement learning layer to let the AI learn from lap times or other metrics.
  • Implement a shadow mode where the AI predicts movements before executing them, similar to how autonomous vehicles test their routes.
  • Encourage community data collection efforts to enhance AI performance.

Related Projects

To explore similar projects, check out:

  • Xbox Game AI – A project using PYXInput for Xbox and PC game control.
  • SerpentAI – A framework to create AIs for any game.
  • Donkey Gym – OpenAI Gym environments for self-driving Donkey Car.
  • AirSim – An Unreal Engine simulator for autonomous vehicles.

Conclusion

At fxis.ai, we believe that such advancements are crucial for the future of AI, as they enable more comprehensive and effective solutions. Our team is continually exploring new methodologies to push the envelope in artificial intelligence, ensuring that our clients benefit from the latest technological innovations.

Now, grab your joystick and get started on your self-driving Mario Kart adventure!

Stay Informed with the Newest F(x) Insights and Blogs

Tech News and Blog Highlights, Straight to Your Inbox