Introduction
In the dynamic realm of natural language processing, we find ourselves at a pivotal moment with the introduction of the NV-Embed-v2 model. As of August 30, 2024, it has ascended to the pinnacle of the Massive Text Embedding Benchmark (MTEB), achieving an impressive score of 72.31 across 56 diverse text embedding tasks. This is not merely an advancement in technology; NV-Embed-v2 represents a significant leap forward, offering a powerful tool that enhances various applications, particularly in retrieval-augmented generation (RAG) systems.
What truly distinguishes NV-Embed-v2 from its predecessors? At its core, this model integrates groundbreaking methodologies, including a sophisticated mechanism whereby the large language model (LLM) engages with latent vectors. This unique interaction facilitates the generation of superior pooled embedding outputs. Furthermore, the implementation of a two-stage instruction tuning process markedly enhances accuracy for both retrieval and non-retrieval tasks. And let’s not overlook the innovative hard-negative mining technique, which effectively minimizes false negatives by leveraging positive relevance scores.
For a more comprehensive exploration of the technical foundations, I invite you to delve into our paper: [NV-Embed: Improved Techniques for Training LLMs as Generalist Embedding Models](https://arxiv.org/pdf/2405.17428).
Model Specifications
- Base Decoder-only LLM: [Mistral-7B-v0.1](https://huggingface.com/istralai/Mistral-7B-v0.1)
- Pooling Type: Latent-Attention
- Embedding Dimension: 4096
Utilizing NV-Embed-v2
Integrating NV-Embed-v2 into your projects is a seamless experience, thanks to its compatibility with Hugging Face Transformers and Sentence Transformers. Below, I will guide you through the process.
Implementation with Hugging Face Transformers
To begin, ensure you have the necessary packages installed. Here’s a quick setup:
pip install torch transformers
Next, let’s encode some queries and passages:
import torch import torch.nn.functional as F from transformers import AutoTokenizer, AutoModel # Define the task and queries task_name_to_instruct = "Given a question, retrieve passages that answer the question." query_prefix = "Instruct: " + task_name_to_instruct + "\nQuery: " queries = [ "Are judo throws allowed in wrestling?", "How to become a radiology technician in Michigan?" ] # Define the passages passages = [ "Since you're reading this, you are probably someone from a judo background or just wondering how judo techniques can be applied under wrestling rules. Yes, judo throws are allowed in freestyle and folkstyle wrestling. Just be careful to follow the slam rules.", "To become a radiologic technologist in Michigan, you need to earn a high school diploma and then an Associate of Applied Science degree. Ensure the program is accredited by the Joint Review Committee on Education in Radiologic Technology (JRCERT)." ] # Load the model with the tokenizer model = AutoModel.from_pretrained("nvidia/NV-Embed-v2", trust_remote_code=True) # Get the embeddings max_length = 32768 query_embeddings = model.encode(queries, instruction=query_prefix, max_length=max_length) passage_embeddings = model.encode(passages, instruction="", max_length=max_length) # Normalize embeddings query_embeddings = F.normalize(query_embeddings, p=2, dim=1) passage_embeddings = F.normalize(passage_embeddings, p=2, dim=1) # Calculate scores scores = (query_embeddings @ passage_embeddings.T) * 100 print(scores.tolist())
Implementation with Sentence Transformers
For those who prefer using Sentence Transformers, the process is equally straightforward:
pip install sentence-transformers
Here’s how to get started:
import torch from sentence_transformers import SentenceTransformer # Define the task and queries task_name_to_instruct = "Given a question, retrieve passages that answer the question." query_prefix = "Instruct: " + task_name_to_instruct + "\nQuery: " queries = [ "Are judo throws allowed in wrestling?", "How to become a radiology technician in Michigan?" ] # Define the passages passages = [ "Yes, judo throws are allowed in freestyle and folkstyle wrestling. Just be careful to follow the slam rules.", "To become a radiologic technologist in Michigan, you need to earn a high school diploma and then an Associate of Applied Science degree." ] # Load the model model = SentenceTransformer("nvidia/NV-Embed-v2", trust_remote_code=True) model.max_seq_length = 32768 model.tokenizer.padding_side = "right" # Get the embeddings batch_size = 2 query_embeddings = model.encode(queries, batch_size=batch_size, prompt=query_prefix, normalize_embeddings=True) passage_embeddings = model.encode(passages, batch_size=batch_size, normalize_embeddings=True) # Calculate scores scores = (query_embeddings @ passage_embeddings.T) * 100 print(scores.tolist())
Licensing Information
It is important to note that NV-Embed-v2 is not intended for commercial use. For a thorough understanding of the licensing terms, please refer to the [license](https://spdx.org/licenses/CC-BY-NC-4.0).
Correspondence
For any inquiries or further information, please do not hesitate to reach out to:
- Chankyu Lee (chankyul@nvidia.com)
- Wei Ping (wping@nvidia.com)
Citation
If you find this model beneficial for your research endeavors, we kindly encourage you to cite our work:
@article{lee2024nv, title={NV-Embed: Improved Techniques for Training LLMs as Generalist Embedding Models}, author={Lee, Chankyu and Roy, Rajarshi and Xu, Mengyao and Raiman, Jonathan and Shoeybi, Mohammad and Catanzaro, Bryan and Ping, Wei}, journal={arXiv preprint arXiv:2405.17428}, year={2024} }
With NV-Embed-v2, we are not merely anticipating the future of text embedding; we are experiencing it. Whether your focus lies in retrieval, classification, or clustering, this model is crafted to elevate your projects to unprecedented levels. Embrace the journey of embedding!
