In an era where 3D reconstruction technology is on the rise, the MobileBrick project stands as a beacon of innovation. This project is led by Kejie Lia, Jia-Wang Bian, Robert Castle, Philip H.S. Torr, and Victor Adrian Prisacariu.

MobileBrick Project Page | arXiv | Dataset

Even 3D scanners can only generate pseudo ground-truth shapes with artifacts. MobileBrick is the first multi-view RGBD dataset, captured on a mobile device, with precise 3D annotations for detailed 3D object reconstruction. We propose a novel data capturing and 3D annotation pipeline in MobileBrick without relying on expensive 3D scanners. The key to creating precise 3D ground-truth shapes is using LEGO models, made from bricks with known geometry. This combination allows a unique opportunity for future research on high-fidelity 3D reconstruction.

Overview

• Install
• Our Dataset
• Evaluation
• Cite
• Changelog

Install

You can install dependencies with Anaconda as follows:

conda env create -f environment.yml
conda activate mobilebrick

Dataset Organisation

The dataset is organized by sequences, with 135 sequences of random shapes used for training, and 18 sequences of meticulously curated LEGO models for evaluation. Understanding the structure of these sequences is akin to assembling a LEGO set itself: each piece holds a certain importance and when combined with others, forms a complete model. Here’s how it’s structured:

• SEQUENCE_NAME
• arkit_depth (depth and confidence maps from ARKit)
• gt_depth (High-resolution depth maps from the aligned GT shape)
• image (The RGB images)
• mask (Object foreground mask)
• intrinsic (Intrinsic matrix of each image)
• pose (Transformation matrix)
• mesh (Ground truth mesh and visibility masks)

Understanding this structure allows you to build a well-formed dataset similar to stacking LEGO bricks in the right order for it to look just as intended.

Evaluation

We provide scripts for evaluating 3D reconstruction and Novel View Synthesis (NVS). Running evaluations can be likened to having different LEGO models and assessing their design and functionality:

python evaluate_3d.py --method $METHOD

For NVS evaluations, use:

python evaluate_nvs.py --method $METHOD

Ensure your reconstruction files are placed correctly in the specified folders to receive accurate results!

Cite

Please cite our work if you find it useful:

latex@article{li2023mobilebrick,
  author = {Kejie Li, Jia-Wang Bian, Robert Castle, Philip H.S. Torr, Victor Adrian Prisacariu},
  title = {MobileBrick: Building LEGO for 3D Reconstruction on Mobile Devices},
  journal = {arXiv preprint arXiv:2303.01932},
  year = {2023}
}

Changelog

• 09032023: MobileBrick is merged into Voxurf.
• 06032023: Dataset is online.

Troubleshooting

If you encounter issues during installation or while running evaluations, consider the following troubleshooting steps:

• Ensure that you have the correct version of Anaconda installed.
• Check if all dependencies are correctly specified in the environment.yml file.
• Double-check your dataset structure to ensure all files are appropriately organized.
• If you face issues related to evaluation scripts, verify that the reconstruction files are in their designated folders.
• Error messages may often help pinpoint the issue—always read them carefully!

For more insights, updates, or to collaborate on AI development projects, stay connected with fxis.ai.

At fxis.ai, we believe that such advancements are crucial for the future of AI, as they enable more comprehensive and effective solutions. Our team is continually exploring new methodologies to push the envelope in artificial intelligence, ensuring that our clients benefit from the latest technological innovations.