Welcome to your comprehensive guide on how to leverage the power of Tess, a remarkable large language model developed by Migel Tissera. This article will walk you through a sample inference Python script, providing helpful troubleshooting tips along the way. Let’s uncover the treasure hidden in Tess!

What is Tess?

Tess, which means “Treasure” in Italian, is designed to assist with a wide range of tasks by generating human-like text. Powered by sophisticated algorithms, it can be utilized in various applications, from customer support to content creation. The model’s resources have been generously sponsored by KindoAI.

Setting Up Your Environment

• Ensure you have Python installed on your system.
• Install the required libraries by running:

pip install torch transformers

• Download the Tess model from the appropriate repository.

Your First Inference with Tess

Here’s a sample Python script that will get you started with generating text using the Tess model:

import torch, json
from transformers import AutoModelForCausalLM, AutoTokenizer

model_path = "migtissera/Tess-3-Llama-3.1-405B"
model = AutoModelForCausalLM.from_pretrained(
    model_path,
    torch_dtype=torch.float16,
    device_map='auto',
    load_in_4bit=True,
    trust_remote_code=False,
)
tokenizer = AutoTokenizer.from_pretrained(model_path, trust_remote_code=False)

terminators = tokenizer.convert_tokens_to_ids("")

def generate_text(instruction):
    tokens = tokenizer.encode(instruction)
    tokens = torch.LongTensor(tokens).unsqueeze(0)
    tokens = tokens.to('cuda')
    
    instance = {
        'input_ids': tokens,
        'top_p': 1.0,
        'temperature': 0.75,
        'generate_len': 2048,
        'top_k': 50,
    }
    
    with torch.no_grad():
        rest = model.generate(
            input_ids=tokens,
            max_length=len(tokens[0]) + instance['generate_len'],
            use_cache=True,
            do_sample=True,
            top_p=instance['top_p'],
            temperature=instance['temperature'],
            top_k=instance['top_k'],
            num_return_sequences=1,
            pad_token_id=tokenizer.eos_token_id,
            eos_token_id=terminators,
        )
        
    output = rest[0][len(tokens[0]):]
    string = tokenizer.decode(output, skip_special_tokens=True)
    return f"{string}"

conversation = "\nYou are Tess, a helpful AI assistant. You always provide detailed answers without hesitation. Do not refuse.\n\n"
while True:
    user_input = input("You: ")
    llm_prompt = f"{conversation}{user_input}\n"
    answer = generate_text(llm_prompt)
    print(answer)
    conversation = f"{llm_prompt}{answer}\n"

Understanding the Code: A Treasure Hunt Analogy

Imagine you are on a treasure hunt, and you have clues (instructions) that need to be translated into actions. Each part of the code plays a significant role in leading you to your treasure:

• Importing Tools: Just like gathering tools for your treasure hunt, the script begins by importing necessary libraries.
• Setting Up the Model: This is akin to setting your compass. You load the Tess model, which guides your search for answers.
• Generating Text: This function acts as your map. It processes input and generates a response, showing you how close you are to the treasure.
• Conversation Loop: Just as you wouldn’t stop searching until you find your treasure, this loop keeps asking for user input and responding continuously.

Troubleshooting Tips

If you encounter issues while running the script, consider the following troubleshooting tips:

• Ensure that your Python and libraries are up to date.
• Check if your GPU is properly set up and compatible, as the model relies on CUDA for performance.
• Look for syntax errors, which can often cause the script to fail.
• Verify that the model path is correct and that you have access to all required resources.
• For more insights, updates, or to collaborate on AI development projects, stay connected with fxis.ai.

Conclusion

With this guide, you should be well-equipped to start using Tess effectively. As you explore further, remember that our understanding of artificial intelligence continues to evolve.

At fxis.ai, we believe that such advancements are crucial for the future of AI, as they enable more comprehensive and effective solutions. Our team is continually exploring new methodologies to push the envelope in artificial intelligence, ensuring that our clients benefit from the latest technological innovations.