Welcome to the world of EXAONE-3.0-7.8B-Instruct! In this guide, we will walk you through the concise yet powerful steps to get started with this bilingual (English and Korean) generative model. With an impressive 7.8 billion parameters and highly competitive benchmark performance, getting to know EXAONE is an exciting venture!

What You’ll Need

• Python installed on your machine
• Transformers library version 4.41 or later
• PyTorch library for model loading

Getting Started with EXAONE

Follow these steps to harness the power of EXAONE:

Step 1: Install Required Libraries

Run the following command to install the transformers and torch libraries:

pip install transformers torch

Step 2: Load the Model

Now it’s time to load the model and tokenizer in your Python environment. Here’s how:

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained(
    "LG-AI/EXAONE-3.0-7.8B-Instruct", 
    torch_dtype=torch.bfloat16, 
    trust_remote_code=True, 
    device_map="auto"
)
tokenizer = AutoTokenizer.from_pretrained("LG-AI/EXAONE-3.0-7.8B-Instruct")

Step 3: Choose Your Prompt

Before you generate text, you’ll need to set your prompt — this is akin to setting the stage for a performance. Here’s how you can define both English and Korean prompts:

prompt = "Explain who you are"  # English example
# For Korean, you could add: prompt = "당신이 누구인지 설명하십시오."
messages = [
    {"role": "system", "content": "You are EXAONE model from LG AI Research, a helpful assistant."},
    {"role": "user", "content": prompt}
]

Step 4: Generate Output

In this last step, we prepare the input and generate the output:

input_ids = tokenizer.apply_chat_template(
    messages, 
    tokenize=True, 
    add_generation_prompt=True, 
    return_tensors="pt"
)
output = model.generate(
    input_ids.to("cuda"),
    eos_token_id=tokenizer.eos_token_id,
    max_new_tokens=128
)

print(tokenizer.decode(output[0]))

Performance Evaluation

The EXAONE model has demonstrated remarkable performance through various benchmarks. It’s designed to engage in meaningful conversations and produce high-quality content. However, it’s essential to keep an eye on the results to ensure they meet your expectations!

Troubleshooting

If you face any issues during the setup or execution of the code, here are some helpful tips:

• Model Loading Issues: Ensure you have a stable internet connection while downloading the model, and check that you’re using the correct model identifier.
• CUDA Not Available: If you’re unable to utilize CUDA for GPU acceleration, try setting ‘device_map’ in the model loading command to ‘cpu’.
• Memory Errors: If you encounter memory-related errors, consider reducing the number of tokens generated by adjusting max_new_tokens.
• For more insights, updates, or to collaborate on AI development projects, stay connected with fxis.ai.

Conclusion

At fxis.ai, we believe that such advancements are crucial for the future of AI, as they enable more comprehensive and effective solutions. Our team is continually exploring new methodologies to push the envelope in artificial intelligence, ensuring that our clients benefit from the latest technological innovations.

Limitations of EXAONE

It’s crucial to note that despite its capabilities, the EXAONE model may sometimes produce inappropriate or biased content. To minimize risks, users should refrain from malicious activities and always use the model ethically.

Final Thoughts

Now that you are equipped with the knowledge to use EXAONE-3.0, venture forth and explore its capabilities. Whether generating text or answering queries, the potential is at your fingertips!