Are you intrigued by the cutting-edge technology powering today’s AI-driven language agents? Welcome to our exploration of Lemur, an open-source language model designed to enhance both natural language processing and coding capabilities. Let’s dive into how to harness the potential of Lemur in your projects!

Why Lemur?

In the ocean of open-source models, the majority specialize in either natural language or coding. However, Lemur elegantly fuses these strengths. Imagine Lemur as a two-part recipe: first, it is pre-trained with a rich dataset of 90 billion tokens, leaning heavily towards coding to establish a solid base. Then, it undergoes instruction tuning over 300,000 examples encompassing both text and code, enhancing its abilities for real-world applications.

Quickstart

Ready to unleash the power of Lemur? Follow these steps to get started.

Setup

• Begin by installing the required dependencies. Open your terminal and follow the commands:

bash
conda create -n xchat python=3.10
conda activate xchat
conda install pytorch==2.0.1 pytorch-cuda=11.8 -c pytorch -c nvidia
conda install -c nvidia label cuda-11.8.0 cuda-nvcc

• Next, install the xchat package:

bash
git clone git@github.com:OpenLemur/Lemur.git
cd Lemur
pip install -e .

Lemur-70B Model Usage

Let’s visualize using the Lemur-70B model as baking a cake. You need your ingredients (the model) and the recipe (the code) to create the final delicacy (output). Here’s how to integrate it:

python
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("OpenLemur/lemur-70b-v1")
model = AutoModelForCausalLM.from_pretrained("OpenLemur/lemur-70b-v1", device_map="auto", load_in_8bit=True)

# Text Generation Example
prompt = "The world is"
input = tokenizer(prompt, return_tensors="pt")
output = model.generate(**input, max_length=50, num_return_sequences=1)
generated_text = tokenizer.decode(output[0], skip_special_tokens=True)
print(generated_text)

# Code Generation Example
prompt = "def factorial(n):\n   if n == 0:\n      return 1"
input = tokenizer(prompt, return_tensors="pt")
output = model.generate(**input, max_length=200, num_return_sequences=1)
generated_code = tokenizer.decode(output[0], skip_special_tokens=True)
print(generated_code)

Lemur-70B-Chat Model Usage

Think of interacting with Lemur-70B-Chat as having a conversation with a wise friend. Here’s a simple way to tap into this model:

python
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("OpenLemur/lemur-70b-chat-v1")
model = AutoModelForCausalLM.from_pretrained("OpenLemur/lemur-70b-chat-v1", device_map="auto", load_in_8bit=True)

# Text Generation Example
prompt = "im_start system\nYou are a helpful, respectful, and honest assistant.\nim_end\nim_start user\nWhat's a lemur's favorite fruit?\nim_end"
input = tokenizer(prompt, return_tensors="pt")
output = model.generate(**input, max_length=50, num_return_sequences=1)
generated_text = tokenizer.decode(output[0], skip_special_tokens=True)
print(generated_text)

# Code Generation Example
prompt = "im_start system\nBelow is an instruction that describes a task. Write a response that appropriately completes the request.\nim_end\nim_start user\nWrite a Python function to merge two sorted lists into one sorted list without using any built-in sort functions.\nim_end"
input = tokenizer(prompt, return_tensors="pt")
output = model.generate(**input, max_length=200, num_return_sequences=1)
generated_code = tokenizer.decode(output[0], skip_special_tokens=True)
print(generated_code)

Evaluation and Performance

To evaluate Lemur’s efficacy, a wide range of datasets were employed, testing its foundational and interactive capabilities across diverse tasks.

Troubleshooting

While enjoying Lemur’s features, you may encounter challenges. Here are some common troubleshooting steps:

• Ensure all dependencies are correctly installed and the environment is activated.
• Double-check that you are referencing the correct model names when loading the tokenizer and model.
• For issues related to model performance, experiment with changing the `max_length` and `num_return_sequences` parameters in generation functions.
• If you encounter inconsistent outputs, consider fine-tuning your prompts for clearer context.

For more insights, updates, or to collaborate on AI development projects, stay connected with fxis.ai.

Conclusion

At fxis.ai, we believe that such advancements are crucial for the future of AI, as they enable more comprehensive and effective solutions. Our team is continually exploring new methodologies to push the envelope in artificial intelligence, ensuring that our clients benefit from the latest technological innovations.