The tf_efficientnetv2_b3 model is a powerful tool designed for image classification tasks, trained on the ImageNet-1k dataset. In this article, we will explore how to utilize this model in Python using the timm library. We’ll not only cover image classification but also delve into feature map extraction and generating image embeddings.

Model Overview

The tf_efficientnetv2_b3 model, a part of the EfficientNet family, boasts impressive performance with:

• Parameters: 14.4 million
• GMACs: 1.9
• Activations: 10.0 million
• Training Image Size: 240 x 240
• Testing Image Size: 300 x 300

You can refer to the original paper here and the official implementation here.

Image Classification

Here is how you can perform image classification using the EfficientNet-v2 model.

python
from urllib.request import urlopen
from PIL import Image
import timm

# Load the image
img = Image.open(urlopen('https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/beignets-task-guide.png'))

# Create the model
model = timm.create_model('tf_efficientnetv2_b3.in1k', pretrained=True)
model = model.eval()

# Get model specific transforms (normalization, resize)
data_config = timm.data.resolve_model_data_config(model)
transforms = timm.data.create_transform(**data_config, is_training=False)

# Classify the image
output = model(transforms(img).unsqueeze(0))  # unsqueeze single image into batch of 1
top5_probabilities, top5_class_indices = torch.topk(output.softmax(dim=1) * 100, k=5)

This code imports the necessary libraries, loads an image from the URL, and preprocesses it for classification. The model outputs the top 5 predicted classes along with their probabilities.

Feature Map Extraction

To extract feature maps from the model, you can modify the model creation code slightly:

python
model = timm.create_model('tf_efficientnetv2_b3.in1k', pretrained=True, features_only=True)
model = model.eval()

# Classify the image
output = model(transforms(img).unsqueeze(0))  # unsqueeze single image into batch of 1

for o in output:
    # Display shape of each feature map in output
    print(o.shape)

The difference here is setting features_only=True. The output will provide the shapes of each feature map, allowing you to understand how the model processes the image.

Image Embeddings

You can also utilize the model to generate image embeddings by doing the following:

python
model = timm.create_model('tf_efficientnetv2_b3.in1k', pretrained=True, num_classes=0)  # remove classifier
model = model.eval()

# Generate embeddings
output = model(transforms(img).unsqueeze(0))  # output is (batch_size, num_features) shaped tensor

By removing the classifier, the model outputs a representation of the image instead of the class probabilities, which can be used for various downstream tasks.

Troubleshooting

If you encounter issues while using the model, here are a few troubleshooting tips:

• Check Dependencies: Ensure you have the necessary libraries installed, including timm, torch, and PIL.
• Image URL: Verify that the image URL is accessible and valid. If the image cannot be loaded, the model will not work.
• Device Compatibility: Ensure your environment supports GPU usage for faster inference. If using a CPU, the processing may be slower.

For more insights, updates, or to collaborate on AI development projects, stay connected with fxis.ai.

Conclusion

At fxis.ai, we believe that such advancements are crucial for the future of AI, as they enable more comprehensive and effective solutions. Our team is continually exploring new methodologies to push the envelope in artificial intelligence, ensuring that our clients benefit from the latest technological innovations.