Welcome to the guide for utilizing the Phi-3 Portuguese Tom Cat 4K Instruct Model. This powerful text generation model, trained with a superset of 300,000 instructions in Portuguese, aims to bridge the gap in models available for the Portuguese language. Let’s delve into how you can harness its capabilities!

Getting Started

Before using the Phi-3 model, ensure you have the following installed on your system:

• Transformers Library
• Accelerate Library
• BitsAndBytes Library

You can install these dependencies by running the following commands:

python
!pip install -q -U transformers
!pip install -q -U accelerate
!pip install -q -U bitsandbytes

Loading the Model

The process of loading the model can be compared to setting up a high-performance racing car; you need to ensure every component is properly in place to achieve optimal speed and efficiency. In this analogy:

• The model is your racing car.
• The tokenizer is like your gear shift, enabling the model to understand and generate language smoothly.

Here’s how to load the Phi-3 model:

python
from transformers import AutoModelForCausalLM, AutoTokenizer

# Load the model and tokenizer
model = AutoModelForCausalLM.from_pretrained('rhaymisonphi-3-portuguese-tom-cat-4k-instruct', device_map=':0')
tokenizer = AutoTokenizer.from_pretrained('rhaymisonphi-3-portuguese-tom-cat-4k-instruct')

model.eval()

Using the Model

After successfully loading the model, it’s time to get it to produce text—think of it as firing up your racing car’s engine and getting it on the track!

python
from transformers import pipeline

# Create a text generation pipeline
pipe = pipeline(
    "text-generation",
    model=model,
    tokenizer=tokenizer,
    do_sample=True,
    max_new_tokens=512,
    num_beams=2,
    temperature=0.3,
    top_k=50,
    top_p=0.95,
    early_stopping=True,
    pad_token_id=tokenizer.eos_token_id,
)

# Create a function to format prompts
def format_template(question: str):
    system_prompt = "Abaixo está uma instrução que descreve uma tarefa, juntamente com uma entrada que fornece mais contexto. Escreva uma resposta que complete adequadamente o pedido."
    return f"{system_prompt}\n\nUsuário: {question}\nAssistente:"

# Generate a response
question = format_template("É possível ir de carro dos Estados Unidos até o Japão?")
response = pipe(question)
print(response)

Troubleshooting Common Issues

While using the Phi-3 model, you may encounter some common issues, just like a race car driver might face during a race. Here are some solutions:

• Out of Memory (CUDA error): If you face a memory problem, consider using 4-bit or 8-bit quantization.
• Performance Issues: Make sure that you are using the correct model based on your hardware capabilities (A100 for full model, T4 for 4-bit, and L4 for 8-bit).
• Inconsistent Output: Ensure that your prompts are clear and contain verbs to guide the model effectively.

For more insights, updates, or to collaborate on AI development projects, stay connected with fxis.ai.

Conclusion

The Phi-3 Portuguese Tom Cat model offers impressive text generation capabilities, essential for bridging linguistic gaps. By following the steps outlined in this blog, you can efficiently deploy the model for various applications in the Portuguese language.

At fxis.ai, we believe that such advancements are crucial for the future of AI, as they enable more comprehensive and effective solutions. Our team is continually exploring new methodologies to push the envelope in artificial intelligence, ensuring that our clients benefit from the latest technological innovations.