The MiniCPM-Reranker is a powerful bilingual cross-lingual text re-ranking model that excels in managing queries in both Chinese and English. Developed by ModelBest Inc., THUNLP, and NEUIR, it leverages the capabilities of the MiniCPM architecture to deliver outstanding results in text classification tasks. In this guide, we will walk you through the steps to use the MiniCPM-Reranker effectively.
Understanding the MiniCPM-Reranker
The MiniCPM-Reranker is like a librarian who knows multiple languages. Imagine you walk into a library with questions about various topics written in different languages. This innovative model sifts through documents in both Chinese and English, determining which ones best answer your query, just as a librarian would pull the most relevant books or articles for you.
Key Features of MiniCPM-Reranker
- Bilingual re-ranking capabilities: Works seamlessly with texts in both Chinese and English.
- Cross-lingual functionality: Can effectively link and re-rank documents across different languages.
- Efficient: Trained on approximately 6 million examples with a maximum input token capacity of 1024.
Usage Instructions
Input Format
The MiniCPM-Reranker accepts inputs in the following format:
sInstruction: instruction
Query: query s documentFor example:
sInstruction: Given a claim about climate change, retrieve documents that support or refute the claim.
Query: However the warming trend is slower than most climate models have forecast. s(document omitted)Requirements
To utilize MiniCPM-Reranker, ensure you have the following dependencies installed:
- transformers==4.37.2
- flash-attn==2.3.5
Running a Demo
Using Huggingface Transformers
Below is a sample code to run the MiniCPM-Reranker:
from transformers import AutoModel, LlamaTokenizer, AutoModelForSequenceClassification
import torch
import numpy as np
class MiniCPMRerankerLLamaTokenizer(LlamaTokenizer):
def build_inputs_with_special_tokens(self, token_ids_0, token_ids_1=None):
if token_ids_1 is None:
return super().build_inputs_with_special_tokens(token_ids_0)
bos = [self.bos_token_id]
sep = [self.eos_token_id]
return bos + token_ids_0 + sep + token_ids_1
model_name = "openbmb/MiniCPM-Reranker"
tokenizer = MiniCPMRerankerLLamaTokenizer.from_pretrained(model_name, trust_remote_code=True)
tokenizer.padding_side = "right"
model = AutoModelForSequenceClassification.from_pretrained(model_name, trust_remote_code=True, attn_implementation="flash_attention_2", torch_dtype=torch.float16).to("cuda")
@torch.no_grad()
def rerank(input_query, input_docs):
tokenized_inputs = tokenizer([[input_query, input_doc] for input_doc in input_docs], return_tensors="pt", padding=True, truncation=True, max_length=1024)
for k in tokenized_inputs:
tokenized_inputs[k] = tokenized_inputs[k].to("cuda")
outputs = model(**tokenized_inputs)
score = outputs.logits
return score.float().detach().cpu().numpy()
queries = []
passages = [["beijing", "shanghai"]]
INSTRUCTION = "Query: "
queries = [INSTRUCTION + query for query in queries]
scores = []
for i in range(len(queries)):
print(queries[i])
scores.append(rerank(queries[i], passages[i]))
print(np.array(scores))Troubleshooting
- If the output does not meet expectations, check that the input format strictly adheres to the specified instructions.
- Ensure all dependencies are installed and are of the correct version.
- For any compatibility issues, verify that the model files are downloaded correctly and that you have the proper permissions.
- If you encounter errors related to GPU usage, ensure CUDA is properly set up on your machine.
- For more insights, updates, or to collaborate on AI development projects, stay connected with fxis.ai.
At fxis.ai, we believe that such advancements are crucial for the future of AI, as they enable more comprehensive and effective solutions. Our team is continually exploring new methodologies to push the envelope in artificial intelligence, ensuring that our clients benefit from the latest technological innovations.
