Welcome to the exciting world of Large Action Models (LAMs)! In this blog, we will guide you through the necessary steps to access the xLAM-7b-fc-r model, understand its capabilities, and troubleshoot common issues. Whether you’re a developer, researcher, or AI enthusiast, this guide is crafted to facilitate your journey into the realm of automated decision-making AI!
Table of Contents
Model Series
The xLAM models come in various sizes tailored for different applications, particularly optimized for function-calling:
| Model | # Total Params | Context Length | Download Model | Download GGUF files |
|---|---|---|---|---|
| xLAM-1b-fc-r | 1.35B | 16384 | 🤗 Link | 🤗 Link |
| xLAM-7b-fc-r | 6.91B | 4096 | 🤗 Link | 🤗 Link |
The `fc` models are geared towards quick and accurate function-calling responses based on input queries. They are lightweight enough to run on everyday devices.
Repository Overview
This repository emphasizes the xLAM-7b-fc-r model, which excels in function-calling tasks. It’s perfect for integrating with diverse applications like weather updates or social media management. Here’s a streamlined analogy for understanding how this model works:
Think of the xLAM model as a highly skilled chef in a bustling restaurant. The chef receives orders (queries) from customers (users) and has a menu of dishes (APIs) to choose from. Depending on the order, the chef selects the appropriate ingredients (API parameters) and prepares the dish (executes the API call) to meet the customer’s request with high efficiency. Just as the chef must adhere to specific recipes to ensure great taste, the xLAM model must follow structured JSON output to deliver accurate responses.
Benchmark Results
The xLAM-7b-fc-r model performed exceedingly well on the Berkeley Function-Calling Leaderboard, achieving:
- 3rd place with an overall accuracy of 88.24%
- Top performance for a model with fewer than 2 billion parameters
Usage
Basic Usage with Huggingface
To start using the xLAM model, you need to install the Transformers library:
pip install transformers>=4.41.0Here’s a brief example to show you how to perform function-calling tasks with the model.
import json
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
torch.random.manual_seed(0)
model_name = "Salesforce/xLAM-7b-fc-r"
model = AutoModelForCausalLM.from_pretrained(model_name, device_map="auto", torch_dtype="auto", trust_remote_code=True)
tokenizer = AutoTokenizer.from_pretrained(model_name)
task_instruction = """You are an expert in composing functions. You are given a question and a set of possible functions. Based on the question, you will need to make one or more function/tool calls to achieve the purpose. If none of the functions can be used, point it out and refuse to answer. If the given question lacks the parameters required by the function, also point it out."""
format_instruction = """The output MUST strictly adhere to the following JSON format, and NO other text MUST be included.
```json
{
"tool_calls": [
{"name": "func_name1", "arguments": {"argument1": "value1", "argument2": "value2"}}
]
}```"""
query = "What's the weather like in New York in fahrenheit?"
get_weather_api = {
"name": "get_weather",
"description": "Get the current weather for a location",
"parameters": {
"type": "object",
"properties": {
"location": {
"type": "string",
"description": "The city and state, e.g. San Francisco, New York"
},
"unit": {
"type": "string",
"enum": ["celsius", "fahrenheit"],
"description": "The unit of temperature to return"
}
},
"required": ["location"]
}
}
search_api = {
"name": "search",
"description": "Search for information on the internet",
"parameters": {
"type": "object",
"properties": {
"query": {
"type": "string",
"description": "The search query, e.g. 'latest news on AI'"
}
},
"required": ["query"]
}
}
openai_format_tools = [get_weather_api, search_api]
# Helper function to convert openai format tools to our more concise xLAM format
def convert_to_xlam_tool(tools):
if isinstance(tools, dict):
return {
"name": tools["name"],
"description": tools["description"],
"parameters": {k: v for k, v in tools["parameters"].get("properties", {}).items()}
}
elif isinstance(tools, list):
return [convert_to_xlam_tool(tool) for tool in tools]
else:
return tools
# Helper function to build the input prompt for our model
def build_prompt(task_instruction: str, format_instruction: str, tools: list, query: str):
prompt = f"[BEGIN OF TASK INSTRUCTION]\n{task_instruction}\n[END OF TASK INSTRUCTION]\n\n"
prompt += f"[BEGIN OF AVAILABLE TOOLS]\n{json.dumps(xlam_format_tools)}\n[END OF AVAILABLE TOOLS]\n\n"
prompt += f"[BEGIN OF FORMAT INSTRUCTION]\n{format_instruction}\n[END OF FORMAT INSTRUCTION]\n\n"
prompt += f"[BEGIN OF QUERY]\n{query}\n[END OF QUERY]\n\n"
return prompt
# Build the input and start inference
xlam_format_tools = convert_to_xlam_tool(openai_format_tools)
content = build_prompt(task_instruction, format_instruction, xlam_format_tools, query)
messages=[{'role': 'user', 'content': content}]
inputs = tokenizer.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt").to(model.device)
outputs = model.generate(inputs, max_new_tokens=512, do_sample=False, num_return_sequences=1, eos_token_id=tokenizer.eos_token_id)
print(tokenizer.decode(outputs[0][len(inputs[0]):], skip_special_tokens=True))
```
When you run the above code, you can expect a JSON output indicating the weather query for New York in Fahrenheit.
Usage with vLLM
To deploy on vLLM, first install the required packages:
pip install vllm openai argparse jinja2
Sample scripts for vLLM usage are available in the examples folder.
License
The xLAM-7b-fc-r model is provided under the CC-BY-NC-4.0 license, with additional terms specified in the Deepseek license.
Citation
If you find this repository helpful, please credit our paper:
@article{liu2024apigen,
title={APIGen: Automated Pipeline for Generating Verifiable and Diverse Function-Calling Datasets},
author={Liu, Zuxin and others},
journal={arXiv preprint arXiv:2406.18518},
year={2024}
}
Troubleshooting
If you encounter any issues while using the xLAM model, here are some common troubleshooting tips:
- Ensure all dependencies are correctly installed: Make sure you have the correct versions of
transformers, torch, and related libraries.
- Check your environment settings: Ensure you are using a compatible version of Python and that your local environment is set up properly.
- Valid Output: If the model doesn’t return the expected JSON output, verify that you are using the provided prompt format correctly.
For more insights, updates, or to collaborate on AI development projects, stay connected with fxis.ai.
At fxis.ai, we believe that such advancements are crucial for the future of AI, as they enable more comprehensive and effective solutions. Our team is continually exploring new methodologies to push the envelope in artificial intelligence, ensuring that our clients benefit from the latest technological innovations.
