Sai Raj Kishore Perla · Yizhi Wang · Ali Mahdavi-Amiri · Hao (Richard) Zhang

iACM Transactions on Graphics (Proceedings of SIGGRAPH), 2024

arXiv | Project Page

Introduction

In the exciting realm of 3D graphics, creating textures for 3D mesh models from a single image is a complex yet fascinating challenge. Our project, EASI-Tex, pioneers an innovative approach by employing a diffusion model to smartly condition a texture transfer, preserving the geometry of the 3D mesh while respecting the details of the source image. This post will guide you through the setup and usage of EASI-Tex in a user-friendly manner.

Setup Environment

To get started, you will need to set up the environment on Ubuntu 20.04 LTS with PyTorch 2.0.1 and CUDA 11.8. Follow the steps below for a seamless setup:

• First, create and activate the conda environment:

conda create -n easitex python=3.10
conda activate easitex

• Next, install PyTorch 2.0.1:

conda install pytorch==2.0.1 torchvision==0.15.2 torchaudio==2.0.2 pytorch-cuda=11.8 -c pytorch -c nvidia

• Then, install the runtime dependencies for PyTorch3D:

conda install -c fvcore -c iopath -c conda-forge fvcore iopath
conda install -c bottler nvidiacub

• Finally, install necessary packages:

pip install -r requirements.txt

• Download necessary model files and place them in the specified directories as mentioned in the README.

Usage

Now that you have set up the environment, let’s dive into the usage of EASI-Tex. We will take a look at a demo and how to work with your data.

Try the Demo

To ensure your setup works perfectly, conduct a sanity check by transferring the texture from a round bird to a cow mesh:

bash run.sh

After the synthesis is complete, check the output files in the outputs directory. You should see a created textured mesh resembling the source image!

Working with Your Data

For best results, ensure that your mesh is triangulated with the following criteria:

• +Y is up
• +Z is the front

You can pre-process your mesh using software like Meshlab or Blender.

For texture transfer, prepare an RGB image and a corresponding mask, ensuring that both files are in the same directory and named correctly.

Troubleshooting

If you encounter any issues or your output isn’t as expected, consider the following:

• Ensure your mesh meets the pre-requisites mentioned above.
• Confirm that all required files were downloaded properly and placed in the correct folders.
• Verify your conda environment is activated and dependencies are correctly installed.

For more insights, updates, or to collaborate on AI development projects, stay connected with fxis.ai.

Conclusion

Our innovative EASI-Tex method, which utilizes a diffusion model, transforms the texture transfer process, allowing for seamless integration of textures onto various 3D meshes. By adhering to the setup and usage instructions outlined, you can explore the fascinating world of single-image mesh texturing with ease.

At fxis.ai, we believe that such advancements are crucial for the future of AI, as they enable more comprehensive and effective solutions. Our team is continually exploring new methodologies to push the envelope in artificial intelligence, ensuring that our clients benefit from the latest technological innovations.