Welcome to the fascinating world of 3D semantic scene completion! In this guide, we’re diving into how to set up and run MonoScene, a powerful tool for monocular 3D processing, developed by Anh-Quan Cao and Raoul de Charette at Inria, Paris, France, and presented at CVPR 2022.

Table of Contents

• Preparing MonoScene
• Running MonoScene
• Inference & Visualization
• Related Camera-Only 3D Occupancy Prediction Projects
• License

Preparing MonoScene

Installation

To get started, you’ll need to set up the necessary environment:

• First, create a conda environment:

$ conda create -y -n monoscene python=3.7
$ conda activate monoscene

• This project runs on Python 3.7, so be sure to install the correct version of PyTorch:

$ conda install pytorch==1.7.1 torchvision==0.8.2 torchaudio==0.7.2 cudatoolkit=10.2 -c pytorch

• Install additional dependencies:

$ cd MonoScene
$ pip install -r requirements.txt

• Then, install the Intel TBB:

$ conda install -c bioconda tbb=2020.25

• Lastly, install MonoScene:

$ pip install -e .

Datasets

You will need to download specific datasets for semantic completion:

• SemanticKITTI: Download from the SemanticKITTI website.
• NYUv2: Fetch the dataset from here.
• KITTI-360: Download the perspective images from the KITTI-360 website.

Running MonoScene

Training

To train MonoScene, you will utilize various scripts, as follows:

For SemanticKITTI:

$ cd MonoScene
$ python monoscenescripts/train_monoscene.py dataset=kitti enable_log=true kitti_root=$KITTI_ROOT kitti_preprocess_root=$KITTI_PREPROCESS kitti_logdir=$KITTI_LOG n_gpus=4 batch_size=4

For NYUv2:

$ cd MonoScene
$ python monoscenescripts/train_monoscene.py dataset=NYU NYU_root=$NYU_ROOT NYU_preprocess_root=$NYU_PREPROCESS logdir=$NYU_LOG n_gpus=2 batch_size=4

Evaluating

To evaluate the performance of your models, you can run:

$ cd MonoScene
$ python monoscenescripts/eval_monoscene.py dataset=kitti kitti_root=$KITTI_ROOT kitti_preprocess_root=$KITTI_PREPROCESS n_gpus=1 batch_size=1

Inference & Visualization

Inference

After training, you can generate outputs and visualize them. Make sure you have a directory ready for storing the outputs:

export MONOSCENE_OUTPUT=pathtomonosceneoutput

Visualization

Mayavi is the primary tool for visualizing the data. Follow these steps:

• Install Mayavi following the official installation guide.
• If you encounter issues with Mayavi, consider using an alternative visualization with Open3D.

Related Camera-Only 3D Occupancy Prediction Projects

Explore more projects that enhance occupancy prediction, including:

• NDC-Scene
• OG: Equip vision occupancy with instance segmentation
• FB-OCC
• Symphonize 3D Semantic Scene Completion

Troubleshooting

Here are some common troubleshooting steps:

• Ensure all dependencies are properly installed.
• If you encounter runtime errors, try updating your conda environment or reinstalling packages.
• If you’re having trouble with Mayavi, check the installation guide or use Open3D as an alternative.
• If you have specific issues, reach out to the community or project maintainers.

For more insights, updates, or to collaborate on AI development projects, stay connected with fxis.ai.

In Conclusion

At fxis.ai, we believe that such advancements are crucial for the future of AI, as they enable more comprehensive and effective solutions. Our team is continually exploring new methodologies to push the envelope in artificial intelligence, ensuring that our clients benefit from the latest technological innovations.