How to Use HorizonNet for Room Layout Estimation

Welcome to our guide on HorizonNet, the innovative approach to learning room layouts using a one-dimensional representation. In this article, we will navigate through the workings of HorizonNet and help you set it up for your projects with ease!

What’s Inside HorizonNet?

HorizonNet is a pure Python implementation that supports several features:

• Inference on images to generate cuboid or general shaped room layouts.
• A 3D layout viewer to visualize your results.
• Correct rotation pose adjustment for Manhattan alignment.
• Pano stretch augmentation to adapt for your tasks.
• Quantitative evaluation metrics, including 2D IoU, 3D IoU, corner error, and pixel error.
• Preparation for your custom dataset and training procedures.

How to Get Started

Installation

To start using HorizonNet, you will need to install Pytorch, which is machine dependent. The tested configuration is:

• Pytorch version: 1.8.1
• Python version: 3.7.6

Ensure you also install the necessary dependencies, which include:

• numpy
• scipy
• sklearn
• Pillow
• tqdm
• tensorboardX
• opencv-python=3.1
• pylsd-nova
• open3d=0.7
• shapely

Preparing the Dataset

HorizonNet requires specific datasets for training and validation, such as:

• Preprocessed PanoContext Stanford2D3D Dataset
• Structured3D Dataset – follow the [tutorial](https://github.com/sunset1995/HorizonNet/blob/master/README_ST3D.md#dataset-preparation) to prepare your data.
• Zillow Indoor Dataset – see the [tutorial](https://github.com/sunset1995/HorizonNet/blob/master/README_ZInD.md#dataset-preparation) to prepare it.

Running Inference on Your Images

Step 1: Pre-processing the Images

Use the following command to align the camera’s rotation pose:

bash
python preprocess.py --img_glob assets/demo.png --output_dir assets/preprocessed

After executing this command, you’ll see the output files in the specified output directory.

Step 2: Estimating Layout with HorizonNet

Now predict the layout using your aligned image:

bash
python inference.py --pth ckpt/resnet50_rnn__mp3d.pth --img_glob assets/preprocessed/demo_aligned_rgb.png --output_dir assets/inferenced --visualize

Check the designated output folder to find visualized results and layout corner data.

Step 3: Visualizing in 3D

To visualize the predicted layout in 3D, run the following:

bash
python layout_viewer.py --img assets/preprocessed/demo_aligned_rgb.png --layout assets/inferenced/demo_aligned_rgb.json --ignore_ceiling

Interact with the viewer using the mouse to change the viewpoint!

Custom Dataset Training

If you wish to train your dataset, see the detailed options by running:

python train.py -h

Hope this sets you on the right path!

Troubleshooting Tips

If you encounter any issues during installation or while running the commands, here are some troubleshooting ideas:

• Ensure all dependencies are correctly installed.
• Verify the paths provided for images and output directories are accurate.
• Check if your Pytorch version matches the requirements.
• If errors arise from data paths, make sure your dataset structure aligns with HorizonNet’s expectations.
• For more complex scenarios, reviewing the project’s GitHub issues page could provide insights.

For more insights, updates, or to collaborate on AI development projects, stay connected with fxis.ai.

Conclusion

At fxis.ai, we believe that such advancements are crucial for the future of AI, as they enable more comprehensive and effective solutions. Our team is continually exploring new methodologies to push the envelope in artificial intelligence, ensuring that our clients benefit from the latest technological innovations.