Are you ready to dive into the world of image-to-image translation? With the official PyTorch implementation of PITI (Pretraining is All You Need for Image-to-Image Translation), you can harness pretraining power across various tasks effortlessly. Buckle up as we guide you through the setup, inference, and some troubleshooting tips!

Introduction

Image-to-image translation is like an artist transforming a rough sketch into a masterpiece. PITI provides a simple and universal framework that enables this transformation with ease. Interested in seeing it in action? Try our online demo for a taste of what this tool can do!

Setup Instructions

Installation

First things first, let’s get started with the installation process.

• Clone the PITI repository:

git clone https://github.com/PITI-Synthesis/PITI.git

• Navigate into the project directory:

cd PITI

Environment Setup

It’s crucial to create an isolated environment to avoid dependency conflicts:

conda env create -f environment.yml

Pretrained Models

Download the pretrained models to kickstart your translation tasks:

• Base Model – Mask-to-Image (Trained on COCO)
• Upsample Model – Mask-to-Image (Trained on COCO)
• Base Model – Sketch-to-Image (Trained on COCO)
• Upsample Model – Sketch-to-Image (Trained on COCO)

If you face any difficulties accessing these links, a backup is available here.

Preparing Images

To test your models, you need images in the right format.

For COCO Dataset

• Download images and annotations from the COCO webpage.
• For mask-to-image, utilize semantic maps in RGB format. Use the script below to generate semantic maps:

python .preprocess/preprocess_mask.py

• For sketch-to-image, use HED to extract sketches. Generate them with:

python .preprocess/preprocess_sketch.py

Inference

Interactive Inference

Want to see results quickly? The interactive GUI can help. To set it up, run:

pip install gradio

Then, execute the following script for an interactive experience:

python inference.py

Batch Inference

For processing numerous images, modify and run:

bash sample.sh

Make sure to include the required arguments as specified in the documentation.

Training

Preparation

Before training, ensure you have preprocessed datasets. Follow the same steps as discussed in the image preparation section.

Start Training

To finetune your models, adjust the respective scripts and execute:

• For the base model:

bash mask_finetune_base.sh

• For the upsample model:

bash mask_finetune_upsample.sh

Troubleshooting

If you encounter issues during installation or inference, consider the following tips:

• Ensure your environment is activated using `conda activate `.
• Check your model paths; ensure they point to the correct directories.
• If gradio doesn’t display the GUI, verify the installation with `pip show gradio`.

For more insights, updates, or to collaborate on AI development projects, stay connected with fxis.ai.

Conclusion

At fxis.ai, we believe that such advancements are crucial for the future of AI, as they enable more comprehensive and effective solutions. Our team is continually exploring new methodologies to push the envelope in artificial intelligence, ensuring that our clients benefit from the latest technological innovations.