Welcome to the exciting world of Qwen1.5, the beta version of Qwen2! As a transformer-based decoder-only language model, Qwen1.5 offers a plethora of features and improvements that elevate your natural language processing (NLP) projects. In this guide, we will take you through the essential steps to get started with Qwen1.5 and troubleshoot any potential issues you might encounter.

What’s New in Qwen1.5?

The Qwen1.5 model brings several upgrades compared to its predecessor, such as:

• Multiple model sizes: Choose among 0.5B, 1.8B, 4B, 7B, 14B, 32B, and 72B dense models.
• Enhanced performance: Experiences a significant boost in chat model preferences.
• Multilingual capabilities: Supports both base and chat models in different languages.
• Context length: Reliable support for a 32K context length across all sizes.
• No requirement for `trust_remote_code`.

For more comprehensive details, refer to our blog post and GitHub repo.

Model Details

Qwen1.5 consists of a series of decoder language models of varying sizes. Here are some key features:

• Built on Transformer architecture with advanced mechanisms like SwiGLU activation and group query attention.
• Includes a tokenizer that adapts to various natural languages and coding languages.
• Post-training optimization through supervised finetuning and direct preference optimization.

How to Install Qwen1.5

To start working with Qwen1.5, ensure you have the latest versions of the required libraries. We recommend installing:

• transformers>=4.37.0

If you don’t meet these requirements, you may encounter the following error:

KeyError: 'qwen2'

Quickstart: Running Qwen1.5

Let’s dive into a quick code snippet to illustrate how to load the tokenizer and model, and how to generate content using Qwen1.5:

from transformers import AutoModelForCausalLM, AutoTokenizer

device = "cuda"  # the device to load the model on
model = AutoModelForCausalLM.from_pretrained(
    "Qwen/Qwen1.5-14B-Chat",
    torch_dtype="auto",
    device_map="auto"
)
tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen1.5-14B-Chat")

prompt = "Give me a short introduction to large language model."
messages = [
    {"role": "system", "content": "You are a helpful assistant."},
    {"role": "user", "content": prompt}
]
text = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True
)
model_inputs = tokenizer([text], return_tensors="pt").to(device)
generated_ids = model.generate(
    model_inputs.input_ids,
    max_new_tokens=512
)
generated_ids = [
    output_ids[len(input_ids):] for input_ids, output_ids in zip(model_inputs.input_ids, generated_ids)
]
response = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0]

Understanding the Code through Analogy

Think of Qwen1.5 as a well-organized kitchen, ready for you to whip up a culinary masterpiece. Each component of the code snippet plays a distinct role in preparing your dish:

• Importing Libraries: Just like gathering your cooking tools, you bring in necessary libraries to assist in your tasks.
• Loading the Model: Imagine this as preheating your oven; you set the model to get ready to generate text.
• Preparing the Recipe: The prompt is your recipe card, guiding you on what to create.
• Cooking the Meal: The text generation is similar to cooking; you mix the ingredients (inputs) and let it transform into your final dish (output).

Tips for Troubleshooting

If you run into issues such as code switching or undesirable output, here are some tips to rectify them:

• Utilize the hyper-parameters found in generation_config.json to refine your outputs.
• Ensure that your libraries are up-to-date with the required versions.

For more insights, updates, or to collaborate on AI development projects, stay connected with fxis.ai.

Conclusion

Qwen1.5-14B-Chat represents a significant leap in language modeling capabilities, and by following the steps above, you can easily integrate it into your NLP workflows. At fxis.ai, we believe that such advancements are crucial for the future of AI, as they enable more comprehensive and effective solutions. Our team is continually exploring new methodologies to push the envelope in artificial intelligence, ensuring that our clients benefit from the latest technological innovations.