In the world of medical imaging and pathology, extracting meaningful features from histology images is pivotal for several downstream applications. Enter H-optimus-0, an impressive 1.1B parameter vision transformer model developed by Bioptimus. With a robust training on over 500,000 HE stained whole slide histology images, H-optimus-0 is an open-source foundation model tailored for the challenges of pathology.

How to Use H-optimus-0 for Feature Extraction

Using H-optimus-0 to extract powerful features is straightforward. The model primarily expects images of size 224×224 pixels that are extracted at a spatial resolution of 0.5 microns per pixel. Below, we’ll walk through the steps required to run inference and obtain the desired features.

python
from huggingface_hub import login
import torch
import timm
from torchvision import transforms

# Login to the Hugging Face hub, using your user access token
# that can be found here:
# https://huggingface.co/settings/tokens

model = timm.create_model(
    'hf-hub:bioptimus/H-optimus-0', pretrained=True, init_values=1e-5, dynamic_img_size=False
)
model.to('cuda')
model.eval()

transform = transforms.Compose(
    [
        transforms.ToTensor(),
        transforms.Normalize(
            mean=(0.707223, 0.578729, 0.703617),
            std=(0.211883, 0.230117, 0.177517)
        ),
    ]
)

input = torch.rand(3, 224, 224)
input = transforms.ToPILImage()(input)

# We recommend using mixed precision for faster inference.
with torch.autocast(device_type='cuda', dtype=torch.float16):
    with torch.inference_mode():
        features = model(transform(input).unsqueeze(0).to('cuda'))

assert features.shape == (1, 1536)

The above code snippet is like assembling a custom pizza. Each step corresponds to a topping, with the code layers working harmoniously to blend flavors into a delicious creation. Here’s how each piece intertwines:

• Logging into Hugging Face: Think of this as checking in at your favorite pizza place; once you’re in, you can order all you need.
• Loading the Model: This is like selecting the base for your pizza – you’re prepared to create a feast with the right foundation.
• Transforming Inputs: Imagine chopping and preparing your ingredients. The transformations ensure your inputs are ready to be served to the model.
• Running Inference: This is where the magic happens, akin to placing your pizza in the oven—you wait for the results to bake to perfection!

Troubleshooting

Sometimes, things might not go as planned when you’re running the code. Here are a few troubleshooting ideas:

• Ensure that you have installed the necessary libraries, particularly torch, timm, and torchvision. If they’re missing, install them using pip:

pip install torch timm torchvision

• If you’re experiencing CUDA-related issues, verify that your system has a compatible NVIDIA GPU and that the CUDA drivers are properly installed. Sometimes, simply rebooting your system can resolve these kinds of issues.
• Double-check the image input sizes. H-optimus-0 expects images precisely sized at 224×224 pixels. Use transforms to resize your images if necessary.
• If your features aren’t being computed, ensure the model is on the correct device (cuda or cpu as necessary), and that your use of mixed precision doesn’t interfere with computations.

For more insights, updates, or to collaborate on AI development projects, stay connected with fxis.ai.

At fxis.ai, we believe that such advancements are crucial for the future of AI, as they enable more comprehensive and effective solutions. Our team is continually exploring new methodologies to push the envelope in artificial intelligence, ensuring that our clients benefit from the latest technological innovations.

Conclusion

H-optimus-0 equips researchers and practitioners with powerful tools for extracting features from histology images, facilitating advancements in mutation prediction, survival analysis, and tissue classification. By following the steps outlined above, you can utilize this fantastic model to enhance your work in medical imaging and pathology.