Welcome to your guide on using the DiTyGemma-2-9b-it model, a fine-tuned version designed specifically for the function calling task using non-synthetic data. This model excels in generating responses based on human-annotated data and is perfect for various conversational AI applications.

Model Overview

The DiTyGemma-2-9b-it model is optimized for handling Function Calling tasks, particularly in Russian. Its data is rich, involving fully annotated inputs which enhance response accuracy significantly.

In addition to safetensors, the model is available in GGUF formats which simplify the process of inference. You only need to download a single file to get started.

Getting Started

• Install Required Libraries: Start by installing the Transformers library via pip:

  pip install -U transformers

Preparing Functions for Function Calling

Before you can use the model, you need to define the functions it will leverage. This can be likened to creating recipes in a cookbook—the model selects which recipe to use to generate its response.

def get_weather(city: str):
    """Function that returns weather information for a specified city."""
    import random
    return "sunny" if random.random() > 0.5 else "rainy"

def get_sunrise_sunset_times(city: str):
    """Function that returns sunrise and sunset times for a specified city."""
    return ["6:00", "18:00"]

Make sure to write these functions in Python with proper documentation strings.

Using the Model

After preparing your functions, import the required libraries, load the model and tokenizer, and then prepare to apply the chat template.

import torch
from transformers import AutoTokenizer, AutoModelForCausalLM

model = AutoModelForCausalLM.from_pretrained(
    "DiTygemma-2-9b-it-russian-function-calling-GGUF",
    device_map="auto",
    torch_dtype=torch.bfloat16,
    cache_dir="PATH_TO_MODEL_DIR"  # optional
)
tokenizer = AutoTokenizer.from_pretrained(
    "DiTygemma-2-9b-it-russian-function-calling-GGUF",
    cache_dir="PATH_TO_MODEL_DIR"  # optional
)

Generating Responses

To generate a response from the model, follow these steps:

• Define your input history messages. Each entry in the list will have a role (system, user, function-call, or function-response) and content.
• Use the tokenizer to apply the chat template to prepare inputs.
• Call the model to generate a response, carefully managing the tokens involved.

history_messages = [
    {"role": "system", "content": "Ты - полезный помощник, имеющий доступ к следующим функциям."},
    {"role": "user", "content": "Привет, не мог бы ты сказать мне, во сколько в Краснодаре восходит солнце?"}
]

inputs = tokenizer.apply_chat_template(
    history_messages,
    tokenize=False,
    add_generation_prompt=True,
    tools=[get_weather, get_sunrise_sunset_times]
)

# Generate response
generated_response = model.generate(inputs)

Troubleshooting

Should you encounter issues, consider the following:

• Ensure that all library dependencies are correctly installed.
• Check that your Python functions are properly defined and imported.
• Verify that the model and tokenizer are correctly loaded from the specified path.
• If errors persist, consult the [Hugging Face documentation](https://huggingface.co/docs) for guidance.

For more insights, updates, or to collaborate on AI development projects, stay connected with fxis.ai.

Conclusion

You should now have a solid understanding of how to work with the DiTyGemma-2-9b-it model for function calling in conversational AI applications. By preparing your functions and using the model correctly, you can generate accurate and helpful responses in Russian.

At fxis.ai, we believe that such advancements are crucial for the future of AI, as they enable more comprehensive and effective solutions. Our team is continually exploring new methodologies to push the envelope in artificial intelligence, ensuring that our clients benefit from the latest technological innovations.