Welcome to the exciting world of Neuralangelo, a powerful tool for high-fidelity neural surface reconstruction. The implementation is built upon the Imaginaire library from NVIDIA’s Deep Imagination Research Group, and this guide will walk you through the steps to set it up, prepare your data, and run the reconstruction process!

Step 1: Installation

Neuralangelo offers two convenient methods for setting up your environment.

• Using Docker Images:
  • docker.io/chenhsuanlin/colmap:3.8 is used for running COLMAP and preprocessing data, including the prebuilt COLMAP library (CUDA-supported).
  • docker.io/chenhsuanlin/neuralangelo:23.04-py3 is intended for executing the main Neuralangelo pipeline.
• Using Conda Environment:

  To install Neuralangelo with conda, run the following commands:

  bash
  conda env create --file neuralangelo.yaml
  conda activate neuralangelo
  

  For COLMAP, additional installation options can be found on the COLMAP website.

Step 2: Data Preparation

Before running Neuralangelo, you’ll need to prepare your data. Refer to the Data Preparation guide for detailed instructions. Ensure that camera poses for each extracted frame from your video are correctly set. The code utilizes a JSON format similar to Instant NGP.

Step 3: Run Neuralangelo

To kick off the Neuralangelo process, use the following command:

bash
EXPERIMENT=toy_example
GROUP=example_group
NAME=example_name
CONFIG=projects/neuralangelo/configs/custom/$EXPERIMENT.yaml
GPUS=1  # use 1 for multi-GPU training!
torchrun --nproc_per_node=$GPUS train.py \
    --logdir=logs/$GROUP/$NAME \
    --config=$CONFIG \
    --show_pbar

Some handy notes:

• The framework supports logging with Weights & Biases. Ensure you have a WB account for this.
• Enable WB logging with the --wandb argument, and specify the project name with --wandb_name.
• Override configs via command line (e.g., --optim.params.lr=1e-2).
• Use --checkpoint=CHECKPOINT_PATH to load a specific checkpoint; use --resume to pick up training where you left off.

Step 4: Isosurface Extraction

To extract isosurface meshes, run this command:

bash
CHECKPOINT=logs/$GROUP/$NAME/xxx.pt
OUTPUT_MESH=xxx.ply
CONFIG=logs/$GROUP/$NAME/config.yaml
RESOLUTION=2048
BLOCK_RES=128
GPUS=1  # use 1 for multi-GPU mesh extraction
torchrun --nproc_per_node=$GPUS projects/neuralangelo/scripts/extract_mesh.py \
    --config=$CONFIG \
    --checkpoint=$CHECKPOINT \
    --output_file=$OUTPUT_MESH \
    --resolution=$RESOLUTION \
    --block_res=$BLOCK_RES

When using this command, you can:

• Add --textured to get textures applied to meshes.
• Add --keep_lcc to avoid noise but be cautious as it can also remove thin structures.
• Lower BLOCK_RES to manage GPU memory usage.
• Reduce RESOLUTION for smaller mesh sizes.

Troubleshooting

If you encounter issues during your setup or operation, don’t worry; here are some common solutions:

• Q: CUDA out of memory. What should I do?

  A: Neuralangelo can require significant GPU memory. Consider adjusting hyperparameters under model.object.sdf.encoding.hashgrid based on your GPU VRAM. For instance:

  • 8GB: dict_size=20, dim=4
  • 12GB: dict_size=21, dim=4
  • 16GB: dict_size=21, dim=8

  These adjustments may reduce reconstruction quality. If training works but you get out of memory during evaluation, lower the image size and batch size in data.val.

• Q: My dataset reconstruction doesn’t look right. What can I do?

  A: Check your camera poses using COLMAP, as they may need refinement. Correct bounding regions can be adjusted in the data preprocessing guide, and ensure your video capture settings optimized for clarity.

For more insights, updates, or to collaborate on AI development projects, stay connected with fxis.ai.

Conclusion

At fxis.ai, we believe that advancements in AI like Neuralangelo are groundbreaking, paving the way for more comprehensive solutions in neural surface reconstruction. Our team is continually exploring new methodologies to push the envelope in artificial intelligence, ensuring that our clients benefit from the latest technological innovations.