Welcome to a transformative exploration in the world of style transfer! In this article, we’ll walk through how to implement a cutting-edge technique that enhances the art of style transfer, transitioning from mere pixel manipulation to the innovative use of parameterized brushstrokes. This exciting project showcases new methodologies presented in the paper Rethinking Style Transfer: From Pixels to Parameterized Brushstrokes by Dmytro Kotovenko and collaborators.

Implementations

The style transfer technique can be executed using two versions of TensorFlow: TensorFlow 1 and TensorFlow 2. It is advisable to use the TensorFlow 1 implementation to replicate the results from the paper accurately.

Installation Guide

To get your environment ready, follow these simple steps:

• Clone the repository:

  git clone https://github.com/CompVis/brushstroke-parameterized-style-transfer

  Navigate into the cloned directory using:

  cd brushstroke-parameterized-style-transfer

• Install TensorFlow 1.14: It’s best to include GPU support. If you are using Conda, run the following commands:

conda create --name tf14 tensorflow-gpu==1.14

conda activate tf14

• Install required dependencies:

  pip install -r requirements.txt

Basic Usage

To stylize an image, you will need to write a small Python script as follows:

from PIL import Image
import model

content_img = Image.open("images/content/golden_gate.jpg")
style_img = Image.open("images/style/van_gogh_starry_night.jpg")
stylized_img = model.stylize(content_img, style_img, num_strokes=5000, num_steps=100, content_weight=1.0, style_weight=3.0, num_steps_pixel=1000)
stylized_img.save("images/stylized.jpg")

Alternatively, you can explore the full functionality of the model via Colab.

Understanding the Code Through Analogy

Think of the function we’re using to create styled images like a chef crafting a unique dish. The chef (your script) has two main ingredients: the content image (base flavor) and the style image (spices). The chef carefully blends these two ingredients by controlling several aspects:

• num_strokes: This can be viewed as the number of times the chef scales the flavors—more strokes can spicy up the dish.
• num_steps: Think of this as the cooking time; the longer it simmers, the richer the fusion of flavors.
• content_weight: This defines the prominence of the base flavor; the higher the weight, the more the original dish is preserved.
• style_weight: This adjusts how much spice gets into the dish. A higher style weight means the dish has a stronger kick!
• num_steps_pixel: This demonstrates the refinements made during the plating process; every pixel gets a careful touch!

Drawing App

We have also developed a Streamlit application that allows you to draw curves controlling the brushstroke flow.

Running the Drawing App Locally

To execute the drawing app on your machine:

CUDA_VISIBLE_DEVICES=0 streamlit run app.py

You can also run the app on a remote server and forward the port to your local machine by following this guide.

Running from Colab

If GPUs aren’t available to you, you can still leverage our Colab setup for the drawing app.

Troubleshooting

• Issue: Failed to install TensorFlow?

Solution: Ensure that you’re using the correct version of CUDA and cuDNN compatible with TensorFlow 1.14.

• Issue: Images not loading?

Solution: Double-check the paths of your content and style images. They must be accurately pointed in your script.

• Issue: Unexpected output quality?

Solution: Experiment with different values for num_strokes and style_weight to refine your output.

• Issue: Streamlit app won’t start?

Solution: Confirm that Streamlit is properly installed in your environment.

For more insights, updates, or to collaborate on AI development projects, stay connected with fxis.ai.

Conclusion

The world of style transfer is evolving rapidly, and grasping these new methodologies can open avenues for creativity. At fxis.ai, we believe that such advancements are crucial for the future of AI, as they enable more comprehensive and effective solutions. Our team is continually exploring new methodologies to push the envelope in artificial intelligence, ensuring that our clients benefit from the latest technological innovations.