In the vast universe of AI and programming, working with rerankers is akin to using a magnifying glass to find rare gems among pebbles – you need precision. A reranker refines results by outputting relevance scores based on queries and passages instead of embeddings. In this article, we will navigate through the ocean of rerankers, revealing how to efficiently utilize them while ensuring you stay afloat through troubleshooting tips.

What is a Reranker?

A reranker operates on the principle of evaluating the relevance of a question (query) against a document (passage). Instead of generating embeddings like traditional models, it outputs a similarity score, allowing users to understand how closely related the two inputs are. The scores can be normalized to a [0,1] scale using a sigmoid function, making them easier to interpret.

Model List

Here are some available models for your reranking tasks:

• BAAI/bge-reranker-base – Lightweight and fast for Chinese and English.
• BAAI/bge-reranker-large – An enhanced version for the same languages.
• BAAI/bge-reranker-v2-m3 – Optimized for multilingual contexts.
• BAAI/bge-reranker-v2-gemma – Works well in multilingual contexts and English proficiency.
• BAAI/bge-reranker-v2-minicpm-layerwise – Allows selection of layers to enhance efficiency and performance in multiple languages.

Getting Started with the Reranker

Begin your journey by installing the FlagEmbedding library:

pip install -U FlagEmbedding

Usage

Normal Rerankers

To compute relevance scores with normal rerankers, follow this sample code:

from FlagEmbedding import FlagReranker
reranker = FlagReranker('BAAI/bge-reranker-v2-m3', use_fp16=True)
score = reranker.compute_score(['query', 'passage'])
print(score)

By setting normalize=True, you can map the scores into a [0,1] range as demonstrated below:

score = reranker.compute_score(['query', 'passage'], normalize=True)
print(score)

LLM-Based Rerankers

To leverage the capabilities of large language models, you can use:

from FlagEmbedding import FlagLLMReranker
reranker = FlagLLMReranker('BAAI/bge-reranker-v2-gemma', use_fp16=True)
score = reranker.compute_score(['query', 'passage'])
print(score)

Layerwise Rerankers

Layerwise rerankers allow you to optimize performance further. Here’s how:

from FlagEmbedding import LayerWiseFlagLLMReranker
reranker = LayerWiseFlagLLMReranker('BAAI/bge-reranker-v2-minicpm-layerwise', use_fp16=True)
score = reranker.compute_score(['query', 'passage'], cutoff_layers=[8])
print(score)

Fine-tuning the Reranker

If you desire to customize your reranker, you’ll need to format your training data as follows:

{"query": str, "pos": List[str], "neg":List[str], "prompt": str}

Train the model using torchrun:

torchrun --nproc_per_node {number of gpus} \
-m FlagEmbedding.llm_reranker.finetune_for_instruction.run \
--output_dir {path to save model} \
--model_name_or_path google/gemma-2b \
--train_data ./toy_finetune_data.jsonl \
--learning_rate 2e-4 \
--num_train_epochs 1

Troubleshooting

If you encounter any issues during installation or usage, try the following:

• Ensure Python and pip are up-to-date.
• Check if all required dependencies are installed.
• If the code doesn’t run, verify your input data format.
• For more insights, updates, or to collaborate on AI development projects, stay connected with fxis.ai.

Conclusion

Rerankers are powerful tools in the domain of text classification, improving the precision of search results. By tuning, deploying, and troubleshooting them, you can significantly enhance the relevance of your AI applications.

At fxis.ai, we believe that such advancements are crucial for the future of AI, as they enable more comprehensive and effective solutions. Our team is continually exploring new methodologies to push the envelope in artificial intelligence, ensuring that our clients benefit from the latest technological innovations.