Welcome to your guide on using SmolLM-Instruct, a suite of small but powerful language models designed for versatility in text generation. With models ranging from 135M to 1.7B parameters, SmolLM stands out for its instructional tuning capabilities. In this article, you’ll learn how to install the necessary libraries, implement the model, and what to do if you encounter issues along the way.

What is SmolLM-Instruct?

Before diving into the code, let’s clarify what SmolLM-Instruct is. Imagine it as a tiny yet highly intelligent chef that can help you whip up a variety of recipes based on simple instructions. The SmolLM models were trained on a carefully curated dataset called Cosmo-Corpus, made up of educational texts and synthetic stories. This ensures that our chef, while small, is well-read and knows a lot about baking (and more!).

Installation: Setting Up Your Kitchen

To begin, you need to install the `transformers` library. This library is like the heavy-duty kitchenware you need to get started. You can easily install it using pip:

pip install transformers

Loading the Model: Getting Our Chef Ready

Once you have the kitchenware, it’s time to get our chef ready for action. This part involves importing the necessary classes and loading the model checkpoint. Here’s how you do it:

from transformers import AutoModelForCausalLM, AutoTokenizer

checkpoint = "HuggingFaceTB/SmolLM-1.7B-Instruct"
device = "cuda"  # Use "cpu" if you don't have a GPU

tokenizer = AutoTokenizer.from_pretrained(checkpoint)
model = AutoModelForCausalLM.from_pretrained(checkpoint).to(device)

Understanding the Code: An Analogy

Think of the `checkpoint` variable as selecting a specific recipe book from your kitchen shelf—it’s essential because different books (or models) contain different recipes (or capabilities). The `device` variable is like deciding whether to use an electric mixer (GPU) or a manual whisk (CPU). The `tokenizer` prepares our ingredients (text) in the right format, while `model` is the chef cooking up deliciously generated content.

Generating Text: Asking the Chef for a Recipe

Now that our chef is ready, let’s ask them something to showcase their skills. The following snippet demonstrates how to get a list of steps for baking a chocolate cake from scratch.

messages = [{"role": "user", "content": "List the steps to bake a chocolate cake from scratch."}]
input_text = tokenizer.apply_chat_template(messages, tokenize=False)
inputs = tokenizer.encode(input_text, return_tensors="pt").to(device)

outputs = model.generate(inputs, max_new_tokens=100, temperature=0.6, top_p=0.92, do_sample=True)
print(tokenizer.decode(outputs[0]))

Explanation of the Code Snippet

Here, we’re sending a query to our chef, packed with the necessary instructions. Once the input is formatted and ready, we let the model generate a response. Think of it like whispering your order to a skilled chef who prepares your meal while you anxiously wait to savor your dish!

Limitations: To Keep in Mind

While SmolLM models are impressive, they come with limitations. They are primarily effective in English and should not be seen as infallible sources of truth. Like our chef who occasionally mixes up ingredients, the generated outputs can sometimes be factually inaccurate or biased. Always use your judgment and cross-verify important details.

Troubleshooting: When Things Don’t Go as Planned

If you encounter issues, consider the following:

– Installation Problems: Ensure that you’re using the correct version of Python and the `transformers` library.
– Device Issues: Check whether your system has the necessary GPU support or fall back to CPU execution by changing the `device` variable.
– Performance Troubles: Parameters in the generation method such as `temperature`, `top_p`, and `max_new_tokens` can drastically affect output. Experiment with these settings.

For more troubleshooting questions/issues, contact our fxis.ai data scientist expert team.

Conclusion

And there you have it! You’re now equipped with the knowledge to set up and utilize the SmolLM-Instruct models for generating text effortlessly. Remember to treat generated outputs with caution and verify any critical information you receive. Happy coding and enjoy your text generation adventures!