If you’ve ever wondered how computers can understand the similarity between two sentences or paragraphs, you’re in the right place! In this blog, we’ll explore how to use a sentence-transformers model to tackle tasks like clustering and semantic search. This will be a user-friendly guide to help you dive into the world of sentence embeddings.

What Is a Sentence Transformers Model?

The sentence-transformers model takes sentences or paragraphs and maps them into a 768-dimensional dense vector space. This is akin to taking a complex painting and converting it into a unique code that captures all its essential features. Each sentence gets a specific set of numerical values representing its meaning, making it easier for algorithms to compare and determine similarity.

Getting Started

Ready to get hands-on? Here’s how to use the sentence-transformers model!

Installation

Before diving into the code, you need to ensure that you have the sentence-transformers library installed. You can do this easily with pip:

pip install -U sentence-transformers

Usage with Sentence-Transformers

Once installed, you can begin using the model by following these straightforward steps:

python
from sentence_transformers import SentenceTransformer

sentences = ["This is an example sentence", "Each sentence is converted"]
model = SentenceTransformer(MODEL_NAME)
embeddings = model.encode(sentences)
print(embeddings)

Using HuggingFace Transformers

If you don’t want to rely on the sentence-transformers library, fret not! You can achieve the same results using HuggingFace Transformers. Let’s break down the process using an analogy:

Think of your sentences as fruits and the transformer model as a juicer. When you feed these fruits (sentences) into the juicer (model), it extracts the essential juice (embeddings) while leaving behind the pulp (unnecessary details). Here’s how it’s done:

python
from transformers import AutoTokenizer, AutoModel
import torch

# Mean Pooling - Take attention mask into account for correct averaging
def mean_pooling(model_output, attention_mask):
    token_embeddings = model_output[0]  # First element contains all token embeddings
    input_mask_expanded = attention_mask.unsqueeze(-1).expand(token_embeddings.size()).float()
    return torch.sum(token_embeddings * input_mask_expanded, 1) / torch.clamp(input_mask_expanded.sum(1), min=1e-9)

# Sentences to encode
sentences = ["This is an example sentence", "Each sentence is converted"]

# Load model from HuggingFace Hub
tokenizer = AutoTokenizer.from_pretrained(MODEL_NAME)
model = AutoModel.from_pretrained(MODEL_NAME)

# Tokenize sentences
encoded_input = tokenizer(sentences, padding=True, truncation=True, return_tensors='pt')

# Compute token embeddings
with torch.no_grad():
    model_output = model(**encoded_input)

# Perform pooling (mean pooling)
sentence_embeddings = mean_pooling(model_output, encoded_input['attention_mask'])

print("Sentence embeddings:")
print(sentence_embeddings)

Evaluating Your Model

To understand how effective your model is, you can reference the Sentence Embeddings Benchmark. Here, you will find automated evaluations of the model’s performance.

Troubleshooting Your Implementation

• Error: Model not found: Ensure that the MODEL_NAME variable is correctly set to your desired model name.
• Installation issues: Verify that you have the correct Python version and dependencies installed.
• Performance issues: If your model runs slowly, try reducing the batch size in your DataLoader.

For more insights, updates, or to collaborate on AI development projects, stay connected with fxis.ai.

Model Training Details

The model was trained with a DataLoader consisting of 760 samples using the CosineSimilarityLoss with specific training parameters. It helps in finding the smallest angle between two vectors, thereby emphasizing their similarity.

In Conclusion

At fxis.ai, we believe that such advancements are crucial for the future of AI, as they enable more comprehensive and effective solutions. Our team is continually exploring new methodologies to push the envelope in artificial intelligence, ensuring that our clients benefit from the latest technological innovations.