Welcome to the exciting world of Kolors-Inpainting! This guide will walk you through the process of utilizing the Kolors-Inpainting inference code and weights, ensuring you achieve stunning inpainting results in no time.
📖 Introduction
Kolors-Inpainting is a powerful tool built upon the foundations of the Kolors-Basemodel. Think of it as a talented artist who uses masks to selectively paint over parts of an image that need enhancement.
The model is designed to handle inpainting by featuring an enhanced UNet architecture with five additional input channels—four for the masked image and one for the mask. The process of inpainting can be visualized as if the model is a specialized artist, using diverse techniques for creating a variety of masks to cover up imperfections.
There are numerous ways to generate these masks, akin to an artist choosing from different techniques or styles—be it random, segmented, or rectangular!
Here’s a glimpse of the Kolors-Inpainting results:
📊 Evaluation
To assess the performance of Kolors-Inpainting, we developed a comprehensive test set comprising 200 masked images paired with text prompts. A panel of image experts rated the outcomes based on several criteria—visual appeal, text faithfulness, inpainting artifacts, and overall satisfaction. Much like chefs tasting and critiquing dishes, these experts provided valuable feedback on various inpainting models.
| Model | Average Overall Satisfaction | Average Inpainting Artifacts | Average Visual Appeal | Average Text Faithfulness |
|---|---|---|---|---|
| SDXL-Inpainting | 2.573 | 1.205 | 3.000 | 4.299 |
| Kolors-Inpainting | 3.493 | 0.204 | 3.855 | 4.346 |
*The higher the scores for Average Overall Satisfaction, Average Visual Appeal, and Average Text Faithfulness, the better. Conversely, the lower the score for Average Inpainting Artifacts, the better.*
Check out the comparison results visualized in the image below:
*Kolors-Inpainting employs Chinese prompts, while SDXL-Inpainting uses English prompts.*
🛠️ Usage
Requirements
The installation process is relatively straightforward and closely follows the steps you’d take for the Kolors-BaseModel.
Steps to Set Up Kolors-Inpainting
-
Repository Cloning and Dependency Installation
apt-get install git-lfs git clone https://github.com/Kwai-Kolors/Kolors cd Kolors conda create --name kolors python=3.8 conda activate kolors pip install -r requirements.txt python3 setup.py install -
Weights Download
You can download the necessary weights from the following location:
huggingface-cli download --resume-download Kwai-Kolors/Kolors-Inpainting --local-dir weights/Kolors-Inpainting -
Inference
Lastly, initiate inpainting by executing the following commands. Here, the model will produce images based on the input provided in the prompts:
python3 inpainting/sample_inpainting.py ./inpainting/asset/3.png ./inpainting/asset/3_mask.png 穿着美少女战士的衣服，一件类似于水手服风格的衣服，包括一个白色紧身上衣，前胸搭配一个大大的红色蝴蝶结。衣服的领子部分呈蓝色，并且有白色条纹。她还穿着一条蓝色百褶裙，超高清，辛烷渲染，高级质感，32k，高分辨率，最好的质量，超级细节，景深 python3 inpainting/sample_inpainting.py ./inpainting/asset/4.png ./inpainting/asset/4_mask.png 穿着钢铁侠的衣服，高科技盔甲，主要颜色为红色和金色，并且有一些银色装饰。胸前有一个亮起的圆形反应堆装置，充满了未来科技感。超清晰，高质量，超逼真，高分辨率，最好的质量，超级细节，景深The image will be saved to
scripts/outputs/
🛠️ Troubleshooting
If you encounter any issues during installation or inference, consider the following tips:
- Ensure you have the correct Python version and all dependencies installed.
- Verify your internet connection during the weights download step.
- Check paths for Typos in the command lines used for inference.
If problems persist, feel free to reach out for assistance or explore collaborative opportunities. For more insights, updates, or to collaborate on AI development projects, stay connected with fxis.ai.
At fxis.ai, we believe that such advancements are crucial for the future of AI, as they enable more comprehensive and effective solutions. Our team is continually exploring new methodologies to push the envelope in artificial intelligence, ensuring that our clients benefit from the latest technological innovations.
