Welcome to your friendly guide on diving into the Functionary model, a remarkable language model designed to execute functions and plugins with precision. Think of it as a skilled assistant that knows when to fetch information and when to keep the conversation going without triggering unnecessary actions.

Understanding Functionary: An Analogy

Imagine you have a personal assistant who can multitask—making coffee, booking appointments, and even checking the weather—all while participating in your chat. This is what Functionary does with coding functions. Just as your assistant decides which task to execute based on your needs, Functionary intelligently analyzes when to use functions and when a simple chat response will suffice.

Key Features at a Glance

• Intelligent parallel tool use
• Analyzes outputs from functions to provide relevant responses
• Decides when to refrain from using tools
• One of the best open-source alternatives to GPT-4
• Supports a code interpreter

How to Get Started

To begin using the Functionary model, follow these steps:

Step 1: Setting Up Your Environment

You need to install the required libraries and ensure you have the right model:

from transformers import AutoModelForCausalLM, AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained("meetkai/functionary-small-v2.5", trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained("meetkai/functionary-small-v2.5", device_map="auto", trust_remote_code=True)

Step 2: Defining Your Functions

Next, set up the function you want to use, much like defining a recipe for a dish you want to cook:

tools = [{
    "type": "function",
    "function": {
        "name": "get_current_weather",
        "description": "Get the current weather",
        "parameters": {
            "type": "object",
            "properties": {
                "location": {
                    "type": "string",
                    "description": "The city and state, e.g. San Francisco, CA"
                }
            },
            "required": ["location"]
        }
    }
}]

Step 3: Crafting Your Messages

Prepare your query to the model by setting up the messages:

messages = [{"role": "user", "content": "What is the weather in Istanbul and Singapore respectively?"}]
final_prompt = tokenizer.apply_chat_template(messages, tools, add_generation_prompt=True, tokenize=False)
tokenizer.padding_side = "left"

Step 4: Running the Model

Now that everything is set, you can execute the model and get your output:

inputs = tokenizer(final_prompt, return_tensors="pt").to("cuda")
pred = model.generate_tool_use(**inputs, max_new_tokens=128, tokenizer=tokenizer)
print(tokenizer.decode(pred.cpu()[0]))

Additional Prompt Template

You can also create a prompt template for more complex inquiries:

from openai import OpenAI

client = OpenAI(base_url="http://localhost:8000/v1", api_key="functionary")
client.chat.completions.create(
    model="path/to/functionary/model/",
    messages=[{"role": "user", "content": "What is the weather for Istanbul?"}],
    tools=[{
        "type": "function",
        "function": {
            "name": "get_current_weather",
            "description": "Get the current weather",
            "parameters": {
                "type": "object",
                "properties": {
                    "location": {
                        "type": "string",
                        "description": "The city and state, e.g. San Francisco, CA"
                    }
                },
                "required": ["location"]
            }
        }
    }],
    tool_choice="auto"
)

Troubleshooting Tips

As with any advanced technology, you might face some hiccups. Here are some potential troubleshooting ideas:

• Ensure all packages are up to date. Sometimes, compatibility issues arise from outdated libraries.
• If you’re experiencing performance issues, check your device’s resources. Ensure it has enough memory and processing capacity.
• Examine function definitions closely. Make sure each parameter is defined as expected.
• In case of any unexpected outputs, refer to the detailed documentation available here.
• For more insights, updates, or to collaborate on AI development projects, stay connected with fxis.ai.

Conclusion

At fxis.ai, we believe that such advancements are crucial for the future of AI, as they enable more comprehensive and effective solutions. Our team is continually exploring new methodologies to push the envelope in artificial intelligence, ensuring that our clients benefit from the latest technological innovations.

Now, go ahead and unleash the power of Functionary in your projects. Happy coding!