The BLOOMZ mT0 models are engineered to understand and respond to human instructions across a multitude of languages. In this blog, we’ll explore how to utilize these models effectively and what to do if you encounter any hiccups along the way.

1. Model Summary

The BLOOMZ mT0 is a family of multitask finetuned models that can handle various natural language tasks in multiple languages with zero-shot capabilities. You can learn more by accessing the repository here and reviewing the accompanying paper.

2. Intended Use

The BLOOMZ mT0 model excels at performing tasks expressed in natural language. Here are some examples of prompts that you can try:

• Translate to English: Je t’aime.
• 一个传奇的开端，一个不灭的神话，这不仅仅是一部电影，而是作为一个走进新时代的标签，永远彪炳史册。你认为这句话的立场是赞扬、中立还是批评?
• Suggest at least five related search terms to Mạng neural nhân tạo.
• Write a fairy tale about a troll saving a princess from a dangerous dragon.
• Explain in a sentence in Telugu what is backpropagation in neural networks.

Feel free to share your generations in the Community tab!

3. How to Use the Model

The usage of the BLOOMZ model depends on whether you’re using a CPU, GPU, or 8-bit GPU support. Below, we’ll walk through the steps for each method.

Using a CPU

python
# pip install -q transformers
from transformers import AutoModelForCausalLM, AutoTokenizer

checkpoint = "bigscience/bloomz-3b"
tokenizer = AutoTokenizer.from_pretrained(checkpoint)
model = AutoModelForCausalLM.from_pretrained(checkpoint)

inputs = tokenizer.encode("Translate to English: Je t’aime.", return_tensors="pt")
outputs = model.generate(inputs)
print(tokenizer.decode(outputs[0]))

Using a GPU

python
# pip install -q transformers accelerate
from transformers import AutoModelForCausalLM, AutoTokenizer

checkpoint = "bigscience/bloomz-3b"
tokenizer = AutoTokenizer.from_pretrained(checkpoint)
model = AutoModelForCausalLM.from_pretrained(checkpoint, torch_dtype="auto", device_map="auto")

inputs = tokenizer.encode("Translate to English: Je t’aime.", return_tensors="pt").to("cuda")
outputs = model.generate(inputs)
print(tokenizer.decode(outputs[0]))

Using 8-bit GPU

python
# pip install -q transformers accelerate bitsandbytes
from transformers import AutoModelForCausalLM, AutoTokenizer

checkpoint = "bigscience/bloomz-3b"
tokenizer = AutoTokenizer.from_pretrained(checkpoint)
model = AutoModelForCausalLM.from_pretrained(checkpoint, device_map="auto", load_in_8bit=True)

inputs = tokenizer.encode("Translate to English: Je t’aime.", return_tensors="pt").to("cuda")
outputs = model.generate(inputs)
print(tokenizer.decode(outputs[0]))

4. Limitations

One critical aspect to keep in mind while using these models is prompt engineering. Achieving high performance may require careful construction of the input prompt. Ideally, you should make sure that your inputs are clear and contextually sufficient. For example, without a terminating punctuation mark, the model might attempt to continue the input instead of concluding with a response. Instead of:

Translate to English: Je t’aime

Use:

Translate to English: Je t’aime.

5. Troubleshooting

Encountering issues while using the BLOOMZ models is not uncommon. Here are a few troubleshooting steps:

• Ensure that all required libraries are correctly installed and updated.
• Verify that your prompts are clear and appropriately punctuated.
• If results seem off, try varying the input or providing more context.

For more insights, updates, or to collaborate on AI development projects, stay connected with fxis.ai.

At fxis.ai, we believe that such advancements are crucial for the future of AI, as they enable more comprehensive and effective solutions. Our team is continually exploring new methodologies to push the envelope in artificial intelligence, ensuring that our clients benefit from the latest technological innovations.