In this era of rapid technological advancements, the Yi series models stand out as powerful open-source large language models, trained by 01.AI. This guide offers you a user-friendly approach to getting started with Yi and making the most out of its features.

What is Yi?

Yi is a state-of-the-art bilingual large language model that showcases exceptional capabilities in language understanding, commonsense reasoning, and reading comprehension. Among its many variations, the Yi-34B model has been recognized as one of the strongest models worldwide, holding top ranks against its peers.

How to Use Yi?

Getting up and running with Yi models can be done in several ways. You can choose to deploy Yi locally using pip, Docker, or even explore it using APIs. Here’s a breakdown of how to begin.

Choose Your Path

• Pip Installation
• Docker Installation
• Llama.cpp Installation
• Web Demo

Quick Start – Pip

If you prefer to run Yi models locally using pip, here’s how you can do it:

Step 0: Prerequisites

• Ensure you have Python 3.10 or later installed.

Step 1: Prepare Your Environment

Execute the following commands in your terminal:

git clone https://github.com/01-ai/Yi.git
cd Yi
pip install -r requirements.txt

Step 2: Download the Yi Model

Use the following links to download models:

• Hugging Face
• ModelScope
• WiseModel

Step 3: Perform Inference

Create a file named quick_start.py with the following code:

from transformers import AutoModelForCausalLM, AutoTokenizer

model_path = 'your-model-path'
tokenizer = AutoTokenizer.from_pretrained(model_path, use_fast=False)
model = AutoModelForCausalLM.from_pretrained(model_path, device_map='auto', torch_dtype='auto').eval()

messages = [{'role': 'user', 'content': 'hi'}]
input_ids = tokenizer.apply_chat_template(messages, tokenize=True, add_generation_prompt=True, return_tensors='pt')
output_ids = model.generate(input_ids.to('cuda'))

response = tokenizer.decode(output_ids[0][input_ids.shape[1]:], skip_special_tokens=True)
print(response)

Run your script to see the output:

python quick_start.py

You should receive a friendly greeting in return!

Quick Start – Docker

For Docker users, here’s how to run Yi-34B-Chat:

Step 0: Prerequisites

• Make sure Docker is installed.
• Install NVIDIA container toolkit for GPU support.

Step 1: Run Docker

docker run -it --gpus all -v your-model-path:/model ghcr.io/01-ai/yi:latest

Step 2: Perform Inference

The steps are similar to the pip method, except you will specify your model path as a volume in Docker.

Web Demo

If you don’t want to deploy Yi locally, you can play with it via:

• Hugging Face (No registration required).

Troubleshooting

While using Yi, you may encounter issues. Here are some troubleshooting tips:

• Ensure that your Python version is up to date.
• Check for any package installation issues in your environment.
• If using Docker, verify that your NVIDIA drivers are correctly installed.
• For any unexpected model outputs, adjust the generation parameters like temperature.

For more insights, updates, or to collaborate on AI development projects, stay connected with fxis.ai.

Conclusion

With the Yi models at your fingertips, you’re equipped to dive into the world of AI development. Explore, experiment, and enhance your projects with these groundbreaking tools.

At fxis.ai, we believe that such advancements are crucial for the future of AI, as they enable more comprehensive and effective solutions. Our team is continually exploring new methodologies to push the envelope in artificial intelligence, ensuring that our clients benefit from the latest technological innovations.