Text Embeddings Inference (TEI) provides a seamless toolkit for deploying and serving open-source text embeddings and sequence classification models. Whether you’re a seasoned developer or a curious beginner, this guide will walk you through the essential steps to get started and troubleshoot potential issues along the way.

Table of Contents

• Get Started
• Supported Models
• Docker
• API Documentation
• Local Install
• Examples

Get Started

To begin using TEI, you need to understand the supported models and deployment options. The toolkit supports various models optimized for embedding and sequence classification, including Nomic, BERT, and many others. You can choose between local installations or using Docker for deployment.

Supported Models

TEI currently supports an extensive range of models designed for different tasks:

• Text Embedding Models: Nomic, BERT, CamemBERT, XLM-RoBERTa, and others.
• Sequence Classification Models: CamemBERT and XLM-RoBERTa, suitable for tasks like sentiment analysis and re-ranking.

For a comprehensive list of the models available, refer to the Massive Text Embedding Benchmark (MTEB) Leaderboard.

Docker

Using Docker for Text Embeddings Inference makes deployment incredibly efficient. Here’s how you can set it up:

model=BAAIbge-large-en-v1.5
volume=$PWD/data # share a volume to avoid downloading weights every run
docker run --gpus all -p 8080:80 -v $volume:data --pull always ghcr.io/huggingface/text-embeddings-inference:1.5 --model-id $model

After running the above command, you can make requests using the curl command.

curl 127.0.0.1:8080/embed -X POST -d '{"inputs":"What is Deep Learning?"}' -H "Content-Type: application/json"

API Documentation

The API documentation for TEI is available via Swagger UI. You can find it at Swagger API Documentation. This is a useful resource to understand all available endpoints and their functionalities.

Local Install

If you prefer to install TEI locally, follow these steps:

curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh # Install Rust
cargo install --path router

For specific architectures like Apple M1/M2, additional steps are required. Ensure you have the necessary libraries and tools for compatibility.

Examples

Here are a couple of ways to utilize TEI:

• Set up an Inference Endpoint with TEI
• RAG containers with TEI

Troubleshooting Tips

If you encounter issues while using Text Embeddings Inference, here are some troubleshooting ideas:

• Ensure that your CUDA and compatible NVIDIA drivers are properly installed, especially if you’re leveraging GPUs.
• If the Docker container doesn’t start, check if all required images are built and available.
• For API request errors, verify the syntax and headers are correctly set in your request commands.
• For more insights, updates, or to collaborate on AI development projects, stay connected with fxis.ai.

At fxis.ai, we believe that such advancements are crucial for the future of AI, as they enable more comprehensive and effective solutions. Our team is continually exploring new methodologies to push the envelope in artificial intelligence, ensuring that our clients benefit from the latest technological innovations.