Getting Started with the PyTorch Version of CS285 Deep Reinforcement Learning Course

Presented by Dr. Sergey Levin at University of California, Berkeley

Table of Contents

• About the Project
• Main Goals
• Completed So Far
• Getting Started
• Prerequisites
• Usage
• Roadmap
• Contributing
• License
• Contact

About The Project

This project aims to create a PyTorch version of the CS285 course originally implemented in TensorFlow 1. Check the original TensorFlow version here.

Main Goals

• Convert all TensorFlow 1 code to the latest version of PyTorch.
• Update the Mujoco environment compatibility to work with newer versions of Python and libraries.

Completed So Far

• Homework assignments 1 through 4 have been successfully converted to PyTorch.

Getting Started

This project is currently under development. To run the assignments, make sure you have the same libraries employed in the TensorFlow version plus PyTorch.

Prerequisites

• Python = 3.6
• Gym = 0.17
• Mujoco-py = 2.0
• Pytorch = 1.5.1
• TensorboardX
• Matplotlib
• Ipython
• Moviepy
• OpenCV
• Box2d-py

Usage Examples

Instructions for executing each of the assignments are located in the README documents within each homework directory.

Roadmap

For a list of known issues, visit the open issues.

Contributing

Currently, the repository is incompatible with the latest versions of libraries such as TensorFlow and Mujoco-py, making installation a challenge. Here’s a step-by-step guide to help you set up the correct versions of these libraries:

Step-by-step Installation Guide

1. Create a new Conda environment based on Python 3.5 and install necessary libraries:

conda create -n cs285_env python=3.5 matplotlib ipython pytorch=1.5.0
source activate cs285_env

2. Clone this repository.
3. Install mujoco-py:
1. Obtain the Mujoco license key from this website.
2. Create a .mujoco folder in your home directory and copy the mjpro150 directory along with your license key:

mkdir ~/.mujoco
cd location_of_your_license_key
cp mjkey.txt ~/.mujoco
cd this_repo/mujoco
cp -r mjpro150 ~/.mujoco

3. Add the following line to the bottom of your .bashrc file:

export LD_LIBRARY_PATH=~/.mujoco/mjpro150/bin

4. Build and install mujoco-py 1.50.1.1 from this link:

tar -xzf mujoco-py-1.50.1.1.tar.gz
cd mujoco-py-1.50.1.1
python setup.py install

5. Install the remaining libraries using pip:

pip install --user --requirement contribution_requirements.txt

4. Make sure to set up the environment for each homework folder before executing scripts:

cd path_to_hw
pip install -e .

License

This project is distributed under the MIT License. See the LICENSE file for more details.

Contact

For any inquiries, reach out to Erfan Miahi on Twitter: @erfan_mhi or email at miahi@ualberta.com.

Project Link: GitHub Repository

Troubleshooting

If you encounter issues during the setup or execution of the project, here are a few troubleshooting ideas:

• Ensure that you are using the specified versions of libraries; mismatched versions could lead to compatibility issues.
• Double-check the installation paths of Mujoco and ensure the license key is correctly placed.
• If any commands fail, consult the output messages for hints about what’s wrong.

For more insights, updates, or to collaborate on AI development projects, stay connected with fxis.ai.

At fxis.ai, we believe that such advancements are crucial for the future of AI, as they enable more comprehensive and effective solutions. Our team is continually exploring new methodologies to push the envelope in artificial intelligence, ensuring that our clients benefit from the latest technological innovations.