Kirill Mazur · Gwangbin Bae · Andrew J. Davison

CVPR 2024

PAPER

Paper Video Project Page

Getting Started

Installation

To set up the SuperPrimitive project, follow these simple steps:

git clone https://github.com/makezur/super_primitive.git --recursive
cd super_primitive
source install.sh

Note: This setup was primarily tested on Ubuntu 20.04 with a single Nvidia RTX 4090. Make sure your system matches these specifications!

Downloading Checkpoints and Data

To download the required checkpoints and datasets, simply execute the following command:

bash .download.sh

This script will automatically download pre-trained checkpoints for both Surface-Aware Mesh (SAM) and surface normal estimation networks. You’ll also get a replica scene and TUM_fr1 sequences.

N.B. If you face system CUDA version mismatches, remember to adjust the pytorch-cuda version in the installation script.

Running Demo (Structure-from-Motion)

For a minimal example of joint pose and geometry estimation, execute:

python sfm_gui_runner.py --config config/replica_sfm_example.yaml

This command estimates a relative pose between two frames and the depth of the source frame.

Monocular Visual Odometry (TUM)

To run your MonoVO on a TUM sequence, use the following command:

python sfm_gui_runner.py --config config/tumodom_desk.yaml --odom

Evaluation

Evaluate by converting estimated trajectories into the TUM format with these commands:

python convert_traj_to_tum.py --root results/desk_undistort_fin_TIMESTAMP
cd results/desk_undistort_fin_TIMESTAMP
evo_ape tum converted_gt_tum_traj.txt converted_tum_traj.txt -as --plot --plot_mode xy --save_results .res.zip

Depth Completion (VOID)

To evaluate depth completion, refer to these official instructions to download VOID. Use the following command to reproduce the quantitative evaluation:

python evaluate_void.py --dataset PATH_TO_VOID_DATASET

Acknowledgments

Special thanks go to the authors of several codebases that contributed to our project:

• DepthCov
• Bilateral Normal Integration
• Surface Normal Uncertainty
• Segment Anything

Troubleshooting Ideas

If you encounter any issues during installation or execution, consider the following tips:

• Ensure that your system meets the necessary specifications mentioned earlier.
• Check the CUDA version compatibility with PyTorch and make necessary adjustments in the installation script.
• Review the logs for any specific error messages to understand the nature of the problem.

For more insights, updates, or to collaborate on AI development projects, stay connected with fxis.ai.

At fxis.ai, we believe that such advancements are crucial for the future of AI, as they enable more comprehensive and effective solutions. Our team is continually exploring new methodologies to push the envelope in artificial intelligence, ensuring that our clients benefit from the latest technological innovations.