How to Install and Use SUMO-RL for Traffic Signal Control

In the ever-evolving world of artificial intelligence, SUMO-RL stands out as a remarkable tool tailored for enhancing traffic signal control through Reinforcement Learning (RL). This guide will provide you with a user-friendly walkthrough of installing and implementing the SUMO-RL package, designed to simplify RL environments using SUMO.

Overview of SUMO-RL

SUMO-RL is a library that allows developers to interact with traffic simulation environments using SUMO (Simulation of Urban MObility). It emphasizes:

• Ease of use with a simple interface for traffic signal control
• Support for multiagent reinforcement learning
• Compatibility with popular RL libraries such as stable-baselines3 and RLlib
• Customizability of states and rewards

Installation Steps

1. Install the Latest Version of SUMO

Run the following commands in your terminal:

sudo add-apt-repository ppa:sumostable
sudo apt-get update
sudo apt-get install sumo sumo-tools sumo-doc

After installation, you need to set the SUMO_HOME variable:

echo export SUMO_HOME=/usr/share/sumo >> ~/.bashrc
source ~/.bashrc

For an added performance boost of approximately 8 times, declare the following variable:

export LIBSUMO_AS_TRACI=1

Note: Activating this option will prevent running multiple simulations in parallel or using sumo-gui.

2. Install SUMO-RL

You can install the stable release version of SUMO-RL via pip:

pip install sumo-rl

Alternatively, to install the latest unreleased version:

git clone https://github.com/LucasAlegre/sumo-rl
cd sumo-rl
pip install -e .

Understanding MDP – Observations, Actions, and Rewards

In SUMO-RL, each traffic signal agent’s observation can be seen as a unique fingerprint of the traffic environment. Let’s break it down using an analogy:

Imagine you’re a bartender in a busy pub. You need to pay attention to several things: the number of patrons (lane density), how many are waiting for drinks (lane queue), and which drinks are currently popular (the traffic signal phase). All of this information helps you decide what to do next (the next action – serving a drink). Your goal is to provide the best service while minimizing wait times (reward function).

Observation Vector

The default observation consists of a vector:

obs = [phase_one_hot, min_green, lane_1_density,..., lane_n_density, lane_1_queue,..., lane_n_queue]

• phase_one_hot: Indicates which green phase is active.
• min_green: Signal for minimum green time.
• lane_i_density: Ratio of vehicles in a lane vs. capacity.
• lane_i_queue: Ratio of queued vehicles in a lane vs. capacity.

Action Space

The action space is discrete. For instance, each traffic signal agent can pick from several predefined green phase configurations every specified interval (delta_time seconds). Picture it as different drink specials at a bar. Each special draws different customer attention (traffic flow).

Reward Function

The reward is determined by the cumulative vehicle delay adjustment. If we stick with our bartender analogy, think of it as the reduction in customers waiting for drinks over time – less waiting indicates higher satisfaction (reward).

APIs to Use SUMO-RL

1. Gymnasium Single-Agent API

For a single traffic light scenario, you can create a standard Gymnasium environment as shown below:

import gymnasium as gym
import sumo_rl

env = gym.make('sumo-rl-v0',
                net_file='path_to_your_network.net.xml',
                route_file='path_to_your_routefile.rou.xml',
                out_csv_name='path_to_output.csv',
                use_gui=True,
                num_seconds=100000)

obs, info = env.reset()
done = False

while not done:
    next_obs, reward, terminated, truncated, info = env.step(env.action_space.sample())
    done = terminated or truncated

2. PettingZoo Multi-Agent API

When managing multiple traffic lights, use PettingZoo:

import sumo_rl

env = sumo_rl.parallel_env(net_file='nets/RESCO/grid4x4/grid4x4.net.xml',
                            route_file='nets/RESCO/grid4x4/grid4x4_1.rou.xml',
                            use_gui=True,
                            num_seconds=3600)

observations = env.reset()

while env.agents:
    actions = {agent: env.action_space(agent).sample() for agent in env.agents}  # Insert your policy here
    observations, rewards, terminations, truncations, infos = env.step(actions)

Troubleshooting Tips

If you encounter issues, consider the following:

• Ensure SUMO is installed correctly and the SUMO_HOME path is correctly set.
• Check network and route files for their correctness and accessibility.
• Look out for any deprecation warnings from the libraries you are using.
• If you need additional support or insights, for more insights, updates, or to collaborate on AI development projects, stay connected with fxis.ai.

At fxis.ai, we believe that such advancements are crucial for the future of AI, as they enable more comprehensive and effective solutions. Our team is continually exploring new methodologies to push the envelope in artificial intelligence, ensuring that our clients benefit from the latest technological innovations.