In this guide, we will explore how to utilize the EfficientNet image classification model, specifically the tf_efficientnet_b3.ns_jft_in1k, using the TIMM (PyTorch Image Models) library. We will walk through the process of image classification, feature extraction, and obtaining image embeddings step-by-step, making it as simple and user-friendly as possible.

What is EfficientNet?

EfficientNet is a family of convolutional neural networks designed to achieve high accuracy while being efficient in terms of model parameters and computational resources. This specific variant, trained on the ImageNet-1k dataset, incorporates state-of-the-art techniques like Noisy Student semi-supervised learning.

Model Setup

Before diving into the code, make sure you have TIMM installed in your Python environment. You can easily install it via pip:

pip install timm

Image Classification

Let’s start with the image classification process. Follow the steps outlined in the code snippet below:

from urllib.request import urlopen
from PIL import Image
import timm
import torch

# Load Image
img = Image.open(urlopen("https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/beignets-task-guide.png"))

# Create Model
model = timm.create_model('tf_efficientnet_b3.ns_jft_in1k', pretrained=True)
model = model.eval()

# Get model specific transforms
data_config = timm.data.resolve_model_data_config(model)
transforms = timm.data.create_transform(**data_config, is_training=False)

# Perform classification
output = model(transforms(img).unsqueeze(0))  # Unsqueeze single image into batch of 1
top5_probabilities, top5_class_indices = torch.topk(output.softmax(dim=1) * 100, k=5)

In this code:

• We first load an image from a specified URL.
• Then, we create and evaluate the EfficientNet model.
• We retrieve the model-specific transformations.
• Finally, we conduct classification and retrieve the top 5 predicted classes and their probabilities.

Feature Map Extraction

Feature maps are essential for understanding how a model interprets an image. Here’s how to extract feature maps from the model:

model = timm.create_model('tf_efficientnet_b3.ns_jft_in1k', pretrained=True, features_only=True)
model = model.eval()

# Perform feature extraction
output = model(transforms(img).unsqueeze(0))  # Unsqueeze single image into batch of 1

# Print shape of each feature map in output
for o in output:
    print(o.shape)

In this code snippet, we modified our model to return the feature maps:

• The model is created with the argument features_only=True, enabling feature extraction.
• The shapes of the resulting feature maps are printed, providing insight into the intermediate representations.

Image Embeddings

We can also obtain image embeddings, which are the output representations before classification. Use the following code:

model = timm.create_model('tf_efficientnet_b3.ns_jft_in1k', pretrained=True, num_classes=0)  # Remove classifier
model = model.eval()

# Get image embeddings
output = model(transforms(img).unsqueeze(0))  # Output is (batch_size, num_features) shaped tensor
# or equivalently:
output = model.forward_features(transforms(img).unsqueeze(0))

Here:

• We remove the classifier from the model by setting num_classes=0.
• The output represents the image embeddings, capturing important visual features.

Troubleshooting

If you encounter issues during implementation, consider the following troubleshooting tips:

• Ensure that the required libraries (PIL, timm, and torch) are properly installed in your environment.
• Verify the URL from which you’re loading the image is valid, as a broken link can cause runtime errors.
• Check that your model is correctly set to evaluation mode using model.eval() to prevent dropout layers from affecting outputs.
• If you experience memory errors, consider using a smaller image or running the code on a machine with more resources.
• For more insights, updates, or to collaborate on AI development projects, stay connected with fxis.ai.

Conclusion

Now that you have the tools to use the EfficientNet image classification model, feature extraction, and embeddings, you can seamlessly incorporate this powerful model into your projects. At fxis.ai, we believe that such advancements are crucial for the future of AI, as they enable more comprehensive and effective solutions. Our team is continually exploring new methodologies to push the envelope in artificial intelligence, ensuring that our clients benefit from the latest technological innovations.