In the vast world of artificial intelligence, image classification stands out as a crucial capability, allowing machines to interpret and categorize images. One impressive model in this domain is the MaxViT Small TensorFlow version, trained on the ImageNet-1k dataset. In this article, we’re going to explore how to effectively implement this model in Python using the TIMM library.

Getting Started with MaxViT

Before we dive into the code, let’s ensure your environment is prepared. Below are the steps you need to follow:

• Install Python and relevant libraries, including timm, Pillow for image handling, and torch for PyTorch integration.
• Make sure you have access to a good image to test the classification.
• You need an active internet connection to load the model and images from the URLs.

Setting Up Image Classification

The following code snippet demonstrates how to load an image and classify it using the MaxViT Small model:

from urllib.request import urlopen
from PIL import Image
import timm
import torch

img = Image.open(urlopen('https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/beignets-task-guide.png'))
model = timm.create_model('maxvit_small_tf_512.in1k', pretrained=True)
model = model.eval()

data_config = timm.data.resolve_model_data_config(model)
transforms = timm.data.create_transform(**data_config, is_training=False)
output = model(transforms(img).unsqueeze(0))  # Unsqueeze single image into batch of 1
top5_probabilities, top5_class_indices = torch.topk(output.softmax(dim=1) * 100, k=5)

In this code:

• We first import the necessary libraries.
• An image is loaded from a URL using urlopen.
• The MaxViT model is created and set to evaluation mode.
• We then prepare the data transformations needed for classification.
• Finally, the model makes a prediction and returns the top 5 probable classes along with their probabilities.

Feature Map Extraction

Sometimes, you might want to see intermediary feature maps generated by the model. Here’s how you can do that:

model = timm.create_model('maxvit_small_tf_512.in1k', pretrained=True, features_only=True)
model = model.eval()

data_config = timm.data.resolve_model_data_config(model)
transforms = timm.data.create_transform(**data_config, is_training=False)
output = model(transforms(img).unsqueeze(0))  # Unsqueeze single image into batch of 1

for o in output:
    print(o.shape)  # Outputs shape of each feature map

In the above code:

• We change the model setup to only extract features.
• The output from the model will be a series of tensors, each representing a different layer’s response to the input image.

Image Embeddings

For some applications, you may require the raw feature embeddings. Let’s examine how to extract these:

model = timm.create_model('maxvit_small_tf_512.in1k', pretrained=True, num_classes=0)  # No classification output
model = model.eval()

data_config = timm.data.resolve_model_data_config(model)
transforms = timm.data.create_transform(**data_config, is_training=False)

output = model(transforms(img).unsqueeze(0))

This code will provide you with an unpooled output tensor representing the embeddings of the image, which can be useful for clustering, retrieval tasks, or as input to further models.

Troubleshooting Common Issues

While the steps above cover the essentials, you may encounter issues. Here are a few common problems and solutions:

• Import Errors: Ensure all libraries are installed correctly using pip install timm torch pillow.
• Model Loading Issues: Verify your internet connection, as the model is fetched online.
• Image Not Found: Double-check the URL of the image.
• Size Mismatch Errors: Ensure the input size is correctly set to 512 x 512 as required by the model.

For more insights, updates, or to collaborate on AI development projects, stay connected with fxis.ai.

Conclusion

MaxViT has opened new horizons for image classification tasks, balancing convolutional layers with self-attention mechanisms to optimize performance. At fxis.ai, we believe that such advancements are crucial for the future of AI, as they enable more comprehensive and effective solutions. Our team is continually exploring new methodologies to push the envelope in artificial intelligence, ensuring that our clients benefit from the latest technological innovations.