In the ever-evolving world of AI, the integration of models with external data sources can greatly enhance their capabilities. The fine-tuned Mistral 7B Instruct v0.2 now supports direct function calling, enabling more complex tasks and richer interactions. This guide will help you set up and utilize this advanced model effectively.

Features Overview

The revolutionary features of this model include:

• Direct Function Calls: Enables structured function calls to integrate APIs and databases seamlessly into your AI’s conversations, making real-time queries and data retrieval possible.

Getting Started: A Step-by-Step Guide

1. Importing Libraries

Before diving into the functionality, let’s import the necessary libraries:

python
from transformers import AutoModelForCausalLM, AutoTokenizer, TextStreamer

2. Initializing the Model and Tokenizer

To work with Mistral, initialize the model and tokenizer:

python
device = 'cuda'
model = AutoModelForCausalLM.from_pretrained("InterSyncMistral-7B-Instruct-v0.2-Function-Calling")
tokenizer = AutoTokenizer.from_pretrained("InterSyncMistral-7B-Instruct-v0.2-Function-Calling")

3. Creating the Text Streamer

Next, set up a text streamer to manage output formatting:

python
streamer = TextStreamer(tokenizer, skip_prompt=True, skip_special_tokens=True)

4. Defining Tools

Here’s where we define the functions that Mistral can call, similar to specifying the ingredients and instructions for a recipe:

python
tools = [
    {
        "type": "function",
        "function": {
            "name": "get_current_weather",
            "description": "Get the current weather",
            "parameters": {
                "type": "object",
                "properties": {
                    "location": {
                        "type": "string",
                        "description": "The city and state, e.g. San Francisco, CA"
                    },
                    "format": {
                        "type": "string",
                        "enum": ["celsius", "fahrenheit"],
                        "description": "The temperature unit to use."
                    }
                },
                "required": ["location", "format"]
            }
        }
    }
]

5. Setting Up the Messages

Prepare the messages that will guide the conversational AI:

python
messages = [
    {
        "role": "user",
        "content": (
            "You are Mistral with function-calling supported. You are provided with function signatures within tools XML tags."
            " You may call one or more functions to assist with the user query. Don't make assumptions about what values to plug into functions."
            " Here are the available tools:\n"
            "tools\n"
            "ftools\n"
            "toolsnn"
            "For each function call, return a JSON object with the function name and arguments within tool_call XML tags as follows:\n"
            "tool_call\n"
            "arguments: args-dict, name: function-name\n"
            "tool_call"
        )
    },
    {
        "role": "assistant",
        "content": "How can I help you today?"
    },
    {
        "role": "user",
        "content": "What is the current weather in San Francisco?"
    },
]

6. Preparing Model Inputs

With the messages in place, prepare the model inputs:

python
inputs = tokenizer.apply_chat_template(messages, tokenize=True, add_generation_prompt=True, return_tensors='pt')
model_inputs = inputs.to(device)

7. Generating the Response

Finally, generate the model’s response:

python
model.to(device)
generate_ids = model.generate(model_inputs, streamer=streamer, do_sample=True, max_length=4096)
decoded = tokenizer.batch_decode(generate_ids)

Expected Output

When successfully executed, you should see an output structure resembling the following:

python
tool_call
arguments: {
    "location": "San Francisco, CA",
    "format": "celsius"
    "name": "get_current_weather"
}
tool_call

Troubleshooting

If you encounter issues during setup or execution, consider the following troubleshooting tips:

• Ensure all libraries are correctly installed and up to date.
• Check if your device supports CUDA and that it is properly configured.
• Verify the model and tokenizer paths are correct and accessible.
• Confirm that your function definitions align with the expected format.

If problems persist, don’t hesitate to reach out for further assistance. For more insights, updates, or to collaborate on AI development projects, stay connected with fxis.ai.

Conclusion

At fxis.ai, we believe that such advancements are crucial for the future of AI, as they enable more comprehensive and effective solutions. Our team is continually exploring new methodologies to push the envelope in artificial intelligence, ensuring that our clients benefit from the latest technological innovations.