Welcome to the exciting world of racing simulations with TORCS (The Open Racing Car Simulator), an open-source racing environment tailored for deep reinforcement learning (RL). In this blog, we’ll navigate the process of setting up your TORCS environment for RL training, troubleshooting, and enhanced customization, all while ensuring a seamless experience.

Table of Content

• Overview
• Environment Specification
• System Requirements
• IMPORTANT!!
• Installation
• Interface Specification
• Folder Structure
• Basic Usage (on a desktop ubuntu)
• Basic Usage (on a server ubuntu)
• Options Specification
• Multithread Example
• Action Space
• Tracks
• Reward
• System Explanation in a Nutshell
• Agent-Environment Communication
• Training results
• [Optional] Further Customization
• Skip the GUI MANU
• Memory Sharing Details
• Obtain the First Person View
• Customize the Race
• Visual Semantic Segmentation
• Reference

Overview

TORCS is a versatile racing car simulator designed for high customization, ideal for researchers looking to train reinforcement learning models with visual observations. By eliminating the need for a graphical user interface (GUI), it streamlines the process significantly.

Environment Specification

This repository includes:

• Source code for modified TORCS
• A dynamic linking library for RL communication
• Lua training codes
• Installation scripts

System Requirements

To run TORCS, ensure your system meets the following requirements:

• Ubuntu 14.04+ (preferably 16.04)
• Torch
• Luasocket, TorchX, Itorch, NNLua, DPNN, Luaposix, Logroll, Classic, TDS

IMPORTANT!!

If running on a server with Nvidia graphics cards, validate your drivers are installed with the flag --no-opengl-files. This avoids potential issues using xvfb. For Ubuntu 14.04 servers, navigate to cd torcs-1.3.6 .configure --disable-xrandr to prevent installation issues.

Installation

Execute the installation script by running:

sudo ./install.sh

Post-installation, launch TORCS via the command line using:

torcs

If running a server, use:

xvfb-run -s -screen 0 640x480x24 torcs

Interface Specification

The interface of this environment aligns with the official [API](https://github.com/Kaixhin/rlenvs#api).

NOTE: Remember to call cleanUp() after training concludes.

Folder Structure

Maintain the following folder structure:

• TORCS/
  • Env.lua
  • Torcs.lua
  • TorcsContinuous.lua
  • TORCSctrl.so
  • TorcsDiscrete.lua
  • TorcsDiscreteConstDamagePos.lua
• game_config/
  • aaa.xml
  • bbb.xml
  • …

Basic Usage (on a desktop ubuntu)

Here’s how to initiate the environment:

local TorcsDiscrete = require 'TORCS.TorcsDiscrete'
local opt = {server = false, game_config = 'quickrace_discrete_single_ushite-city.xml', use_RGB = false, mkey = 817, auto_back = false}
local env = TorcsDiscrete(opt)
local totalstep = 817
local reward, terminal, state = 0, false, env:start()
repeat 
    repeat 
        reward, observation, terminal = env:step(doSomeActions(state)) 
    until terminal 
    if terminal then 
        reward, terminal, state = 0, false, env:start() 
    end 
until nowstep == totalstep
env:cleanUp()

Basic Usage (on a server ubuntu)

Prior to environment initiation, set up xvfb:

bash xvfb_init.sh 99

Then, run your script:

lua test.lua

Options Specification

Configure the racing car environment by passing an options table:

• server: bool value (true for server mode).
• game_config: string (name of the game configuration file).
• use_RGB: bool value (true to use RGB observations).
• mkey: integer (key for memory sharing).
• auto_back: bool value (false to disable reverse driving).

Multithread Example

Check torcs_test.lua for multithreading implementation.

Action Space

The environment supports two action spaces:

• Discrete Action Space: Defined by discrete actions, allowing specific commands like throttle and brake.
• Continuous Action Space: Capacities allowing between full braking and full throttle adjustments.

This flexibility emulates human-like driving responses in a simulated environment.

Tracks

Customize your racing track through the configuration files found in the game_config folder.

Reward

Modify the reward function by overriding reward() function in your configurations.

System Explanation in a Nutshell

Throughout the running game, a state machine tracks progress, updating physical states and querying drivers for actions based on their parameters like speed and distance.

Agent-Environment Communication

Shared memory facilitates inter-process communication between the environment and the RL model, enabling seamless exchange of racing status and actions.

Training Results

Results from a typical A3C model running on this environment may include impressive metrics such as:

• Game Config: quickrace_discrete_slowmulti_ushite-city.xml
• Environment: TorcsDiscreteConstDamagePos
• Total Steps: 2e7

[Optional] Further Customization

This section outlines advanced modifications, including:

1. How to bypass GUI configurations.
2. Memory sharing intricacies.
3. Generating and utilizing first-person views.
4. Customizing racing settings.
5. Implementing semantic segmentation for visual inputs.

Skip the GUI MANU

Related files include torcs-1.3.6/src/main.cpp and torcs-1.3.6/src/lib/raceengine/client/raceinit.cpp.

Memory Sharing Details

Accessing the shared memory requires modification in several specified files to enable effective data exchanges between the different components of TORCS.

Obtain the First Person View

To capture the first-person view, adjustments must be made in the rendering configurations. This allows for rich observation inputs during training.

Customize the Race

Follow specific procedures to set up custom race configurations to enhance the training experience. A step-by-step guide is provided in the documentation.

Visual Semantic Segmentation

This section describes methods to modify track configurations for optimized semantic output, essential for advancing visual training models.

Reference

The foundational structures of the memory-sharing scheme are adapted from various sources, including DeepDriving from Princeton and custom implementations in the RL environment.

Troubleshooting

If you encounter issues, consider the following troubleshooting tips:

• Ensure all dependencies are installed correctly and versions are compatible.
• Double-check your folder structure for any misconfigurations.
• If experiencing graphical issues, revisit your Nvidia driver installations.

For more insights, updates, or to collaborate on AI development projects, stay connected with fxis.ai.

At fxis.ai, we believe that such advancements are crucial for the future of AI, as they enable more comprehensive and effective solutions. Our team is continually exploring new methodologies to push the envelope in artificial intelligence, ensuring that our clients benefit from the latest technological innovations.

Conclusion

Setting up TORCS for reinforcement learning is a rewarding journey filled with learning opportunities and innovations. Dive in today and explore the vast potential it holds for future AI models!