Welcome to the world of AI and language modeling! In this article, we will guide you through the steps to get started with PersianMind, a remarkable language model developed for Persian and English languages. This model achieves state-of-the-art results on several benchmarks, including the Belebeles benchmark and ParsiNLU tasks. Let’s delve into the process of utilizing this innovative tool.

Model Description

PersianMind is a sophisticated language model designed by an expert team at the University of Tehran. It is licensed for non-commercial use under CC BY-NC-SA 4.0. The model’s creators are:

• Pedram Rostami
• Ali Salemi
• Mohammad Javad Dousti

How to Get Started with the Model

To begin using PersianMind, you will need to install several libraries, including sentencepiece, accelerate, PyTorch, and 🤗Transformers. Once you have these prerequisites set up, you can run the following code:

python
from transformers import AutoTokenizer, AutoModelForCausalLM
import torch

device = 'cuda' if torch.cuda.is_available() else 'cpu'

model = AutoModelForCausalLM.from_pretrained(
    'universitytehran/PersianMind-v1.0',
    torch_dtype=torch.bfloat16,
    low_cpu_mem_usage=True,
    device_map='auto',
)

tokenizer = AutoTokenizer.from_pretrained('universitytehran/PersianMind-v1.0')

TEMPLATE = "contextnYou: {prompt}nPersianMind:"
CONTEXT = "This is a conversation with PersianMind. It is an artificial intelligence model designed by a team of NLP experts at the University of Tehran to help you with various tasks such as answering questions, providing recommendations, and helping with decision making. You can ask it anything you want and it will do its best to give you accurate and relevant information."
PROMPT = "در مورد هوش مصنوعی توضیح بده."

model_input = TEMPLATE.format(context=CONTEXT, prompt=PROMPT)
input_tokens = tokenizer(model_input, return_tensors='pt')
input_tokens = input_tokens.to(device)

generate_ids = model.generate(**input_tokens, max_new_tokens=512, do_sample=False, repetition_penalty=1.1)
model_output = tokenizer.batch_decode(generate_ids, skip_special_tokens=True, clean_up_tokenization_spaces=False)[0]

print(model_output[len(model_input):])

Understanding the Code: An Analogy

Think of working with PersianMind as preparing a meal using a complex recipe. The steps in the recipe (code) are essential for your dish (model output) to turn out perfectly. Here’s how the different parts relate:

• Ingredients List (Imports): Just like you need certain ingredients to cook, you’ll need libraries and modules such as transformers and torch to prepare your model.
• Preparing the Kitchen (Device Setup): Deciding whether you’ll use an oven (GPU) or a stove (CPU) is crucial, similar to choosing your computational device based on availability.
• Cooking Process (Model Setup): Once you have your ingredients, you follow steps to mix them (loading the model and tokenizer) carefully.
• Final Outcome (Generate Output): Finally, after cooking (running the model), you serve the dish (output) and enjoy the flavors (results) of your labor.

How to Quantize the Model

If you’re aiming to run PersianMind on devices with limited resources, model quantization is key. To begin this process, install the bitsandbytes library and use the following code:

python
model = AutoModelForCausalLM.from_pretrained(
    'universitytehran/PersianMind-v1.0',
    device_map='auto',
    low_cpu_mem_usage=True,
    load_in_8bit=True
)

Alternatively, for quantizing the model in 4-bit, use this configuration:

python
from transformers import BitsAndBytesConfig

quantization_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_use_double_quant=True,
    bnb_4bit_quant_type='nf4',
)

model = AutoModelForCausalLM.from_pretrained(
    'universitytehran/PersianMind-v1.0',
    quantization_config=quantization_config,
    device_map='auto'
)

Troubleshooting

While getting started with PersianMind, you might encounter some challenges. Here are some common troubleshooting ideas:

• Dependency Issues: Make sure all required libraries are installed correctly. Run pip install -r requirements.txt if you have a requirements file.
• GPU Not Detected: Ensure you have a compatible GPU installed and the right drivers are updated. Use torch.cuda.is_available() to check for availability.
• Memory Errors: If you experience memory-related errors, consider reducing the size of the model or the input data.
• For more insights, updates, or to collaborate on AI development projects, stay connected with fxis.ai.

Conclusion

At fxis.ai, we believe that such advancements are crucial for the future of AI, as they enable more comprehensive and effective solutions. Our team is continually exploring new methodologies to push the envelope in artificial intelligence, ensuring that our clients benefit from the latest technological innovations.

By following these steps, you’re now equipped to harness the power of PersianMind for your language needs! Happy coding!