Welcome to the world of Pyke Diffusers, a cutting-edge modular library built in Rust for pretrained diffusion model inference. This library is unique as it provides a way to create stunning images via ONNX Runtime, touted for its incredible acceleration capabilities on both CPUs and GPUs.

Features of Pyke Diffusers

• Text-to-image generation for Stable Diffusion v1, v2, and v2.1 based models.
• Optimized for CPU and GPU, 45% faster than PyTorch, while using 20% less memory.
• Memory-efficient tracing allows operations with just 2GB of RAM on mobile devices.
• Advanced textual inversion capabilities that allow for detailed prompt weighting.
• Multiple scheduler implementations, ensuring flexibility in usage.

Prerequisites

Before diving into using Pyke Diffusers, ensure your development environment meets the following prerequisites:

• {@code Rust} v1.62.1 or higher must be installed.
• For CPU usage, a recent x86-64 CPU (Haswell or Zen) is recommended.
• If using CUDA: {@code CUDA v11.6} and {@code cuDNN v8.2.x} are required.
• For TensorRT, the same CUDA version along with {@code TensorRT v8.4} should be present.
• DirectML requires a DirectX 12 compatible GPU and Windows 10 v1903 or later.

Using Pyke Diffusers

To utilize Pyke Diffusers for image generation, follow the steps below:

1. Install the Library

Add the following to your Cargo.toml:

[dependencies]
pyke-diffusers = { git = "https://github.com/pykeio/diffusers", rev = "69c33b996416ca42535e4f524068081b93486770" }

# If you'd like to use CUDA:
pyke-diffusers = { git = "https://github.com/pykeio/diffusers", rev = "69c33b996416ca42535e4f524068081b93486770", features = ["ort-cuda"] }

2. Implement Image Generation

With the dependencies set, now you can initiate the text-to-image inference:

rust
use pyke_diffusers::{EulerDiscreteScheduler, OrtEnvironment, SchedulerOptimizedDefaults, StableDiffusionOptions, StableDiffusionPipeline, StableDiffusionTxt2ImgOptions};

let environment = OrtEnvironment::default().into_arc();
let mut scheduler = EulerDiscreteScheduler::stable_diffusion_v1_optimized_default()?;
let pipeline = StableDiffusionPipeline::new(environment, "stable-diffusion-v1-5", StableDiffusionOptions::default())?;
let mut imgs = StableDiffusionTxt2ImgOptions::default()
    .with_prompt("photo of a red fox")
    .run(pipeline, mut scheduler)?;
imgs.remove(0).into_rgb8().save("result.png")?;

Understanding the Code

Let’s demystify that code snippet with an analogy. Imagine you are a chef:

• You first set the kitchen environment (creating the OrtEnvironment instance).
• Next, you choose a cooking method (the scheduler, in this case, EulerDiscreteScheduler).
• Then you take the recipe book (the pipeline represented by StableDiffusionPipeline).
• Finally, you follow the recipe to make a beautiful dish (generate an image based on the prompt).

And voilà! Your delicious dish is saved for sharing, just like the generated image is saved as {@code “result.png”}.

Examples

Pyke Diffusers includes an interactive demo for Stable Diffusion. You can run the following command:

bash
$ cargo run --example stable-diffusion-interactive --features ort-cuda -- ~pathtostable-diffusion

Check the examples folder for other demonstrations and documentation for more detailed instructions.

Troubleshooting Tips

• If you face issues with CUDA drivers, ensure that they are properly installed and compatible with your version of Rust.
• For ONNX Runtime errors, check if you have copied the required binaries from the target directory.
• Low memory usage issues may indicate that you are not using the correct pipeline for your hardware capabilities. Consider switching to a less memory-intensive pipeline.
• If you encounter errors related to features in Cargo.toml, verify that you have specified the necessary features correctly.
• For more insights, updates, or to collaborate on AI development projects, stay connected with fxis.ai.

At fxis.ai, we believe that such advancements are crucial for the future of AI, as they enable more comprehensive and effective solutions. Our team is continually exploring new methodologies to push the envelope in artificial intelligence, ensuring that our clients benefit from the latest technological innovations.