Welcome to our guide on using the PLaMo-13B-Instruct, an instruct fine-tuned model that excels at generating text in both Japanese and English. In this article, we’ll walk you through the steps to get started with this powerful language model, along with some troubleshooting tips to optimize your experience.

Model Overview

PLaMo-13B-Instruct enhances the capabilities of the original PLaMo-13B model by utilizing an 8192 context length and fine-tuning it with multiple Japanese datasets. Released under the Apache License 2.0, this model supports various applications, including generating natural language responses based on prompts.

Installation Requirements

First, you’ll need to install the required libraries. You can do this through your command line:

sh python -m pip install numpy sentencepiece torch transformers accelerate

Getting Started with PLaMo-13B-Instruct

Once you have the libraries installed, you can start coding. Below is a concise Python script that initializes the model and generates text based on a user-defined prompt.

import torch
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained(
    "pfnet/plamo-13b-instruct",
    trust_remote_code=True,
)
model = AutoModelForCausalLM.from_pretrained(
    "pfnet/plamo-13b-instruct",
    trust_remote_code=True,
    torch_dtype=torch.bfloat16,
    device_map="auto",
)

def completion(prompt: str, max_new_tokens: int = 128) -> str:
    inputs = tokenizer(prompt, return_tensors="pt").to("cuda")
    generated_ids = model.generate(
        inputs.input_ids,
        eos_token_id=2,
        pad_token_id=3,
        max_new_tokens=max_new_tokens,
        temperature=1,
        top_p=0.95,
        top_k=50,
        do_sample=True,
    )
    return tokenizer.decode(generated_ids[0], skip_special_tokens=True, clean_up_tokenization_spaces=True)

def generate_prompt(messages: list) -> str:
    sep = "\n###"
    prompt = [
        "以下はタスクを説明する指示で、文脈を説明した入力とペアになっています。",
        "要求を適切に補完するよう応答を書いてください。",
    ]
    roles = {"instruction": "指示", "response": "応答", "input": "入力"}
    for msg in messages:
        prompt.append(sep + roles[msg['role']] + ":\n" + msg['content'])
    prompt.append(sep + roles['response'] + ":\n")
    return "".join(prompt)

prompt = generate_prompt([
    {"role": "instruction", "content": "日本の首都はどこですか？"},
])
print(completion(prompt, max_new_tokens=128))

Understanding the Code: An Analogy

Imagine you’re a chef in a kitchen, and you want to create a delicious meal using a special recipe. Each component of your process corresponds to a part of the code:

• Importing Libraries: Think of this as gathering all your ingredients (libraries) before starting to cook. You need them all for your dish to be successful.
• Loading the Model: Just as reading the recipe gives you instructions on how to prepare the meal, loading the model prepares the kitchen for creating responses to prompts.
• Defining Functions: These are like different cooking techniques; for instance, one might be to sauté vegetables while another could be to boil water. The functions `completion` and `generate_prompt` represent different methods for handling input and generating outputs.
• Executing the Prompt: Finally, when you serve the dish (or print the output), you present the generated text to the diner (user). This is where all the preparation culminates into a delightful experience.

Troubleshooting Tips

If you encounter any issues, try the following troubleshooting steps:

• Ensure all libraries are correctly installed and updated to their latest versions.
• Check if your GPU is properly configured and recognized by PyTorch.
• For memory-related errors, consider reducing the `max_new_tokens` parameter in the `completion` function.
• Make sure the input prompt is correctly formatted according to the required structure for the model.
• If you have further questions, don’t hesitate to reach out for help or insights. For more insights, updates, or to collaborate on AI development projects, stay connected with fxis.ai.

Conclusion

In conclusion, the PLaMo-13B-Instruct model offers exciting capabilities for text generation tasks. By following the steps outlined in this guide, you can effectively leverage its strengths for various applications. At fxis.ai, we believe that such advancements are crucial for the future of AI, as they enable more comprehensive and effective solutions. Our team is continually exploring new methodologies to push the envelope in artificial intelligence, ensuring that our clients benefit from the latest technological innovations.