Welcome to your guide on leveraging the capabilities of Functionary Small v3.1, a cutting-edge language model designed to interpret and execute functions seamlessly. Whether you are a developer or a tech enthusiast, this blog aims to walk you through setting up and using Functionary effectively.

What is Functionary Small v3.1?

Functionary is a language model that can intelligently interpret commands and execute predefined functions or plugins. Think of it as a smart assistant who knows exactly when to get things done, whether it’s pulling the weather data for multiple cities or assisting with other tasks without overdoing it. Functionary uses a template inspired by Meta’s Llama 3.1, ensuring that it functions efficiently and effectively.

Key Features

• Intelligent parallel tool use
• Capability to analyze function outputs and provide suitable responses based on them
• Smart decision-making on when to utilize tool functions vs. providing direct conversational replies
• An excellent open-source alternative to GPT-4
• Support for code interpretation

Getting Started with Functionary

To harness the power of Functionary, you will need to set it up properly. Below is a step-by-step guide:

Step 1: Install Required Libraries

You need to import the necessary libraries from Transformers. If you haven’t done so yet, you can do this with the following code:

from transformers import AutoModelForCausalLM, AutoTokenizer

Step 2: Load the Model

Load the tokenizer and model using the given identifiers:

tokenizer = AutoTokenizer.from_pretrained('meetkaifunctionary-small-v3.1', trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained('meetkaifunctionary-small-v3.1', device_map='auto', trust_remote_code=True)

Step 3: Define the Tool

Next, you’ll define a tool that the model can use. Imagine this as teaching your assistant how to fetch the weather:

tools = {
    'type': 'function',
    'function': {
        'name': 'get_current_weather',
        'description': 'Get the current weather',
        'parameters': {
            'type': 'object',
            'properties': {
                'location': {
                    'type': 'string',
                    'description': 'The city and state, e.g. San Francisco, CA',
                },
            },
            'required': ['location']
        }
    }
}

Step 4: Create a Message and Final Prompt

Now, prepare your message for the model:

messages = {
    'role': 'user',
    'content': 'What is the weather in Istanbul and Singapore respectively?'
}
final_prompt = tokenizer.apply_chat_template(messages, tools, add_generation_prompt=True, tokenize=False)

Step 5: Execute the Model

Finally, it’s time to run the model!

inputs = tokenizer(final_prompt, return_tensors='pt').to('cuda')
pred = model.generate_tool_use(**inputs, max_new_tokens=128, tokenizer=tokenizer)
print(tokenizer.decode(pred.cpu()[0]))

Understanding the Code: An Analogy

Consider the model as an adept chef in a kitchen. The ingredients (functions) must be prepared (defined) by the sous-chef before the chef starts to cook (execute). The chef must have a clear recipe (instructions from the user) on how to combine those ingredients. With the ability to take ingredients (tools) from the pantry (system prompts) at any moment, the chef ensures the meal is prepared just right without wasting resources. By asking relevant questions about the dishes (weather inquiries) and understanding when to pull off the oven (when to use the functions), the chef guarantees a smooth cooking experience.

Troubleshooting

If you encounter any issues while setting up or running the Functionary model, consider the following tips:

• Ensure that you have a stable internet connection; sometimes the model’s dependencies are fetched online.
• Verify that you have all necessary access permissions, especially when working with remote code.
• Check for any syntax errors in your code; Python is sensitive to indentation and proper use of objects.
• If the model is not responding as expected, double-check that your tools are correctly defined and formatted.
• Use the model’s error messages as clues; they often provide insight into what went wrong.

For more insights, updates, or to collaborate on AI development projects, stay connected with fxis.ai.

In Conclusion

Functionary Small v3.1 embodies the future of AI interaction, combining utility with user-friendly functionality. By following the steps outlined above, you can tap into its power to enhance your projects or workflows.

At fxis.ai, we believe that such advancements are crucial for the future of AI, as they enable more comprehensive and effective solutions. Our team is continually exploring new methodologies to push the envelope in artificial intelligence, ensuring that our clients benefit from the latest technological innovations.