The Multi-view CNN (MVCNN) project aims to learn a general-purpose descriptor that recognizes shapes based on convolutional neural networks (CNN). In this guide, we will walk through the steps to set up and use MVCNN for shape recognition, including troubleshooting tips to help you along the way.

Understanding MVCNN

Picture this: you have a complex sculpture, and your task is to identify it from various angles. MVCNN acts like a highly trained art critic who has seen the sculpture from multiple views, enabling it to describe and recognize the shape effectively. It utilizes different perspectives (or views), such as line drawings and rendered 3D models, without relying on color or texture. These diverse views contribute to creating a comprehensive shape descriptor.

Installation

Before you dive into using MVCNN, you need to set it up on your system:

• Install Dependencies:
  Run the following command to update the interface:

  bash git submodule update --init

• Compile:
  To compile for CPU, you might need to set two environment variables:

  bash
  MATLABDIR=MATLAB_ROOT
  MEX=MATLAB_ROOT/bin/mex
  matlab -nodisplay -r setup(true); exit;

  To compile for GPU (with cuDNN), the command changes slightly:

  bash
  MATLABDIR=MATLAB_ROOT
  MEX=MATLAB_ROOT/bin/mex
  matlab -nodisplay -r setup(true, struct(enableGpu, true, enableCudnn, true)); exit;

  Note: Ensure that your Visual Studio path is correctly set.

Using MVCNN

Once installed, you can start using MVCNN to extract shape descriptors:

• For a Single Shape:

  matlab
  shape_compute_descriptor(bunny.off);

  This command extracts the descriptor for a single shape and saves it in a .txt file.
• For Multiple Shapes in a Folder:

  matlab
  shape_compute_descriptor(my_mesh_folder, useUprightAssumption, false);

  The descriptors will be saved in the same folder as text files.
• Post-processing with Learned Metrics:

  matlab
  shape_compute_descriptor(my_mesh_folder, cnnModel, my_cnn.mat, ... metricModel, my_metric.mat, applyMetric, true);

Downloading Datasets

To train and evaluate your models, download the following datasets and place them under the data folder:

• modelnet40v1 (12 views w/ upright assumption)
• modelnet40v2 (80 views wo/ upright assumption)
• shapenet55v1 (12 views w/ upright assumption)
• shapenet55v2 (80 views wo/ upright assumption)

Running Training Examples

Finally, to run training examples, ensure that LD_LIBRARY_PATH is set correctly, then execute:

bash
LD_LIBRARY_PATH=CUDA_ROOT/lib64:CUDNN_ROOT
matlab -nodisplay -r run_experiments; exit;

Troubleshooting

If you encounter any issues during installation or usage, consider the following:

• Ensure that all paths for MATLAB and kNN implementations are correctly set.
• If experiencing compilation errors, check if MATLAB and related dependencies are installed and updated.
• For issues during shape descriptor extraction, ensure your mesh files are correctly formatted and accessible.
• For more insights, updates, or to collaborate on AI development projects, stay connected with fxis.ai.

At fxis.ai, we believe that such advancements are crucial for the future of AI, as they enable more comprehensive and effective solutions. Our team is continually exploring new methodologies to push the envelope in artificial intelligence, ensuring that our clients benefit from the latest technological innovations.