Welcome to your go-to guide for using the AMD-Llama-135m language model! Whether you’re a novice or an expert, this guide will help you navigate the installation and usage of this powerful tool with confidence.

Introduction

The AMD-Llama-135m model, based on the Llama2 architecture, is specially designed to run on AMD Instinct MI250 accelerators. It boasts a myriad of features, including a smoothly loadable format as LlamaForCausalLM using Hugging Face transformers. The model also employs the Llama2 tokenizer, making it an ideal candidate for speculative decoding in both Llama2 and CodeLlama.

Model Details

Here’s a summary of the AMD-Llama-135m specifications:

• Parameter Size: 135M
• Number of Layers: 12
• Hidden Size: 768
• FFN Intermediate Size: 2048
• Number of Heads: 12
• Attention Type: Multi-Head Attention
• Activation Function: Swiglu
• Context Window Size: 2048
• Vocabulary Size: 32000

Quickstart

Using AMD-Llama-135m is straightforward. Below is a simple example to guide you:

from transformers import LlamaForCausalLM, AutoTokenizer

# Load the model and tokenizer
model = LlamaForCausalLM.from_pretrained('amdAMD-Llama-135m')
tokenizer = AutoTokenizer.from_pretrained('amdAMD-Llama-135m')

# Input text for the model
inputs = tokenizer('Tell me a story?', add_special_tokens=False, return_tensors='pt')
tokens = model.generate(**inputs)
output = tokenizer.decode(tokens[0])
print(output)

You can use this model as an assistant for CodeLlama as well. The steps are similar:

# Load assistant model
assistant_model = LlamaForCausalLM.from_pretrained('amdAMD-Llama-135m-code')
model = LlamaForCausalLM.from_pretrained('codellamaCodeLlama-7b-hf')

inputs = tokenizer('def quick_sort(array):', return_tensors='pt')
tokens = model.generate(**inputs, assistant_model=assistant_model, max_new_tokens=100)
output = tokenizer.decode(tokens[0])
print(output)

Understanding the Code: A Tale of Two Workers

Imagine you’re in a bakery where two bakers, the main one and the assistant, are working on a cake order. The main baker focuses on creating the base while the assistant adds layers and frosting. This is similar to how the AMD-Llama-135m functions. The first block of code initializes the main baker (the model), who generates the content based on the input. The second block introduces the assistant, who specializes in refining the service, effectively aiding in the creation of more refined responses (outputs).

Training Data

The AMD-Llama-135m has undergone extensive training using the SlimPajama and Project Gutenberg datasets, totaling around 670 billion training tokens. An extensive foundation guarantees robust language processing capabilities.

Evaluation

The model’s effectiveness is validated through benchmarks like SciQ, PIQA, and MMLU, establishing its performance in natural language processing tasks.

Troubleshooting

If you encounter issues loading the model or errors in execution, consider the following troubleshooting steps:

• Ensure all dependencies are correctly installed, especially transformers of at least version 4.36.2.
• Check your internet connection; model downloading requires a stable connection.
• Verify that you have sufficient disk space, as both the SlimPajama and Project Gutenberg datasets are sizable.
• Consult Hugging Face’s documentation for additional support.

For more insights, updates, or to collaborate on AI development projects, stay connected with fxis.ai.

Conclusion

At fxis.ai, we believe that such advancements are crucial for the future of AI, as they enable more comprehensive and effective solutions. Our team is continually exploring new methodologies to push the envelope in artificial intelligence, ensuring that our clients benefit from the latest technological innovations.