The MaxViT model is a sophisticated image classification system that has been pretrained on the large ImageNet-21k dataset and fine-tuned on the more specific ImageNet-1k dataset. This creates a powerful tool for processing images in a variety of applications. In this guide, we’ll walk through how to use the MaxViT model effectively.

Understanding MaxViT Model Variants

MaxViT encompasses various model architectures, which can be likened to different vehicles designed for specific terrains:

• CoAtNet: Think of this as a hybrid vehicle, blending the benefits of convolutional blocks in its earlier stages with self-attention transformer blocks for more complex environments.
• MaxViT: This represents a utility vehicle with uniform block setups suitable for all stages, optimized for handling different types of roads securely.
• CoAtNeXt: Picture a sports car: it’s nimble and replaces traditional blocks with ConvNeXt blocks while ensuring effective handling through LayerNorm.
• MaxxViT: Imagine this as an upgraded utility vehicle, incorporating ConvNeXt blocks in place of the original MBConv blocks but still retaining intuitive stability.
• MaxxViT-V2: This is like an ultra-modern utility vehicle, designed not just to get the job done but to make it smooth and efficient with extra features.

Using the MaxViT Model

Image Classification

To utilize the MaxViT model for image classification, follow these steps:

python
from urllib.request import urlopen
from PIL import Image
import timm

img = Image.open(urlopen("https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/beignets-task-guide.png"))
model = timm.create_model("maxvit_large_tf_512.in21k_ft_in1k", pretrained=True)
model = model.eval()

# get model specific transforms (normalization, resize)
data_config = timm.data.resolve_model_data_config(model)
transforms = timm.data.create_transform(**data_config, is_training=False)
output = model(transforms(img).unsqueeze(0))  # unsqueeze single image into batch of 1

top5_probabilities, top5_class_indices = torch.topk(output.softmax(dim=1) * 100, k=5)

Feature Map Extraction

To extract feature maps, the code below can be implemented:

python
from urllib.request import urlopen
from PIL import Image
import timm

img = Image.open(urlopen("https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/beignets-task-guide.png"))
model = timm.create_model("maxvit_large_tf_512.in21k_ft_in1k", pretrained=True, features_only=True)
model = model.eval()

# get model specific transforms (normalization, resize)
data_config = timm.data.resolve_model_data_config(model)
transforms = timm.data.create_transform(**data_config, is_training=False)
output = model(transforms(img).unsqueeze(0))  # unsqueeze single image into batch of 1

for o in output:
    print(o.shape)

Image Embeddings

For generating image embeddings, use the following approach:

python
from urllib.request import urlopen
from PIL import Image
import timm

img = Image.open(urlopen("https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/beignets-task-guide.png"))
model = timm.create_model("maxvit_large_tf_512.in21k_ft_in1k", pretrained=True, num_classes=0)  # remove classifier nn.Linear
model = model.eval()

# get model specific transforms (normalization, resize)
data_config = timm.data.resolve_model_data_config(model)
transforms = timm.data.create_transform(**data_config, is_training=False)
output = model(transforms(img).unsqueeze(0))  # output is (batch_size, num_features) shaped tensor
output = model.forward_head(output, pre_logits=True)  # output is a (1, num_features) shaped tensor

Troubleshooting Tips

• If you experience trouble loading images from URLs, ensure the URL is accessible and correct.
• If your model fails to load or throws errors, check the dependencies (like timm) and make sure you’re using compatible versions of libraries.
• For any performance issues, consider verifying your hardware compatibility, specifically the configuration of your GPU.
• While utilizing the model, if you receive unexpected results, review the preprocessing steps as differences in image size or format can lead to errors in output.

For more insights, updates, or to collaborate on AI development projects, stay connected with fxis.ai.

Conclusion

MaxViT provides a robust solution for image classification with its various architectures and features. Whether you are extracting feature maps or classifying images, the steps outlined above will help streamline your process. At fxis.ai, we believe that such advancements are crucial for the future of AI, as they enable more comprehensive and effective solutions. Our team is continually exploring new methodologies to push the envelope in artificial intelligence, ensuring that our clients benefit from the latest technological innovations.