Welcome to an engaging guide on leveraging the power of the Airavata model, a cutting-edge Hindi instruction-tuned language model developed by AI4Bharat. Whether you are a novice or an experienced developer, this blog will walk you through the steps of setting up the model, its usage, and some troubleshooting techniques to enhance your journey.

What is Airavata?

Airavata is a 7 billion parameter model, fine-tuned on the IndicInstruct dataset. It excels in generating text in Hindi, making it an exceptional tool for tasks that involve instruction and dialogue. The model aims to improve human-computer interaction in the Hindi language through various datasets, including specific challenges designed to test its capabilities.

Setting Up Airavata

To get started, follow these simple steps:

• Clone the code repository by accessing IndicInstruct GitHub.
• Install all the necessary dependencies in your Python environment.
• Download or clone the Airavata model to the same machine where you set up your environment.

Input Format

When you interact with the Airavata model, ensure your input is formatted in a chat format. An example is given below:

user
Your message here!

For optimal results, always include a newline after the assistant’s message. This practice significantly impacts the generation quality!

Fine-tuning Hyperparameters

The Airavata model is fine-tuned using LoRA (Low-Rank Adaptation) with several hyperparameters configured to enhance its performance:

• LoRA Rank: 16
• LoRA alpha: 32
• LoRA Dropout: 0.05
• Epochs: 4
• Learning rate: 5e-4
• Batch Size: 128
• Floating Point Precision: bfloat16

These settings are crucial for achieving the desired accuracy and performance in the model’s tasks.

Example Usage

Here’s an analogy to help you understand the model’s function. Think of Airavata as a highly skilled chef in a large kitchen. When you provide it with specific recipes (input prompts), it will use its knowledge and ingredients to create refined dishes (output). The chef, however, needs the right tools and preparation (proper formatting and input) to produce a meal that meets your expectations.

This is how you can run the inference:

python3
import torch
from transformers import AutoTokenizer, AutoModelForCausalLM

device = "cuda" if torch.cuda.is_available() else "cpu"

def create_prompt_with_chat_format(messages, bos="", eos="", add_bos=True):
    formatted_text = ""
    for message in messages:
        role = message['role']
        content = message['content']
        if role == 'system':
            formatted_text += f"system\n{content}\n"
        elif role == 'user':
            formatted_text += f"user\n{content}\n"
        elif role == 'assistant':
            formatted_text += f"assistant\n{content.strip()}{eos}\n"
        else:
            raise ValueError(f"Invalid role: {role}")
    formatted_text += "assistant\n"
    return bos + formatted_text if add_bos else formatted_text

def inference(input_prompts, model, tokenizer):
    # Code continues...

Running this setup will provide you with the insights from the Airavata model.

Troubleshooting

If you encounter any difficulties while working with the Airavata model, here are a few troubleshooting ideas:

• Ensure your dependencies are correctly installed. Mismatched versions can lead to runtime errors.
• Check the input format. An improper format might cause the model to generate incorrect or irrelevant outputs.
• Confirm your GPU resources if you’re using CUDA-enabled devices; ensure they are set up and accessible.
• If the model fails to run or responds incorrectly, you might want to reset the model weights and reinitialize your setup.

For more insights, updates, or to collaborate on AI development projects, stay connected with fxis.ai.

A Final Word

At fxis.ai, we believe that such advancements are crucial for the future of AI, as they enable more comprehensive and effective solutions. Our team is continually exploring new methodologies to push the envelope in artificial intelligence, ensuring that our clients benefit from the latest technological innovations.