The StableLM 2 12B Chat model is a powerful 12 billion parameter instruction-tuned language model. This model leverages Direct Preference Optimization (DPO) to generate coherent and contextually relevant text. In this blog post, we will guide you through the process of utilizing this model step-by-step.

Getting Started with StableLM 2 12B Chat

Before diving into the usage, make sure you have the correct version of the Transformers library. You will need version 4.40.0 or later.

Installation

• Ensure you have Python installed on your machine.
• Install the transformers library using this command:

pip install transformers==4.40.0

Using the Model

To use the StableLM 2 12B Chat model, you need to follow several steps. Let’s break it down with a fun analogy.

Imagine that creating an application using StableLM 2 is like baking a cake. You need the right ingredients (libraries), a recipe (code), and the baking process (execution) to present a delicious cake (output).

Step 1: Import Necessary Libraries

First, you bring out your ingredients from the pantry:

from transformers import AutoModelForCausalLM, AutoTokenizer

Step 2: Load the Model and Tokenizer

Next, you measure out your flour and sugar—this is where you set up your model:

tokenizer = AutoTokenizer.from_pretrained('stabilityai/stablelm-2-12b-chat')
model = AutoModelForCausalLM.from_pretrained('stabilityai/stablelm-2-12b-chat', device_map='auto')

Step 3: Prepare the Input

Now, it’s time to mix the ingredients. You have to prepare your prompt:

prompt = {'role': 'user', 'content': 'Implement snake game using pygame'}
inputs = tokenizer.apply_chat_template(prompt, add_generation_prompt=True, return_tensors='pt')

Step 4: Generate Output

Once mixed, you bake your cake. Here, you generate the output:

tokens = model.generate(inputs.to(model.device), max_new_tokens=100, temperature=0.7, do_sample=True)
output = tokenizer.decode(tokens[:, inputs.shape[-1]:][0], skip_special_tokens=False)

Step 5: Print the Result

Finally, you present your cake, or in this case, the generated output:

print(output)

Function Calling Capability

StableLM 2 12B Chat can also call functions, which is like decorating your cake with icing. Here’s how to use this feature:

system_prompt = "You are a helpful assistant with access to the following functions."
messages = [
    {'role': 'system', 'content': system_prompt},
    {'role': 'user', 'content': 'Please, generate a picture of the Eiffel Tower at night!'}
]
inputs = tokenizer.apply_chat_template(messages, add_generation_prompt=True, return_tensors='pt')
tokens = model.generate(inputs.to(model.device), max_new_tokens=1024, temperature=0.5, do_sample=True)
output = tokenizer.decode(tokens[:, inputs.shape[-1]:][0], skip_special_tokens=True)
print(output)

Troubleshooting

• If you encounter issues during installation, ensure that your Python version is compatible with the transformers library. Upgrade Python if necessary.
• If the model is returning unexpected outputs, consider tweaking the temperature parameter. A lower value (near 0.2) generally results in more deterministic outputs.
• If you are getting errors related to the model or tokenizer not found, double-check the strings in the from_pretrained() functions to ensure they are correct.

For more insights, updates, or to collaborate on AI development projects, stay connected with fxis.ai.

Conclusion

At fxis.ai, we believe that such advancements are crucial for the future of AI, as they enable more comprehensive and effective solutions. Our team is continually exploring new methodologies to push the envelope in artificial intelligence, ensuring that our clients benefit from the latest technological innovations.