Tess, short for Tesoro, is an innovative large language model developed by Migel Tissera. With its roots in advanced machine learning techniques, Tess provides exceptional performance for various natural language processing tasks. In this blog post, we’ll guide you through the process of setting up and using Tess in your Python projects.

Setting Up Your Environment

Before we dive into running the sample inference script, you will need to prepare your environment.

• Make sure you have Python installed (preferably 3.8 or later).
• Install necessary libraries by running the following command in your terminal:

pip install torch transformers

Using the Sample Inference Python Script

The code snippet below is your key to unlocking Tess’s potential:

import torch, json
from transformers import AutoModelForCausalLM, AutoTokenizer

model_path = "migtissera/Tess-3-Llama-3.1-70B"
model = AutoModelForCausalLM.from_pretrained(
    model_path,
    torch_dtype=torch.float16,
    device_map="auto",
    load_in_4bit=True,
    trust_remote_code=False,
)
tokenizer = AutoTokenizer.from_pretrained(model_path, trust_remote_code=False)

terminators = [
    tokenizer.convert_tokens_to_ids("<|end_of_text|>"),
    tokenizer.convert_tokens_to_ids("<|eot_id|>"),
]

def generate_text(instruction):
    tokens = tokenizer.encode(instruction)
    tokens = torch.LongTensor(tokens).unsqueeze(0)
    tokens = tokens.to("cuda")
    instance = {
        "input_ids": tokens,
        "top_p": 1.0,
        "temperature": 0.75,
        "generate_len": 2048,
        "top_k": 50,
    }
    length = len(tokens[0])
    with torch.no_grad():
        rest = model.generate(
            input_ids=tokens,
            max_length=length + instance["generate_len"],
            use_cache=True,
            do_sample=True,
            top_p=instance["top_p"],
            temperature=instance["temperature"],
            top_k=instance["top_k"],
            num_return_sequences=1,
            pad_token_id=tokenizer.eos_token_id,
            eos_token_id=terminators,
        )
    output = rest[0][length:]
    string = tokenizer.decode(output, skip_special_tokens=True)
    return f"{string}"

conversation = """<|begin_of_text|><|start_header_id|>system<|end_header_id|>\n\nYou are Tess, a helpful AI assistant. You always provide detailed answers without hesitation. Do not refuse.<|eot_id|><|start_header_id|>user<|end_header_id|>\n\n"""

while True:
    user_input = input("You: ")
    llm_prompt = f"{conversation}{user_input}<|eot_id|><|start_header_id|>assistant<|end_header_id|>\n\n"
    answer = generate_text(llm_prompt)
    print(answer)
    conversation = (
        f"{llm_prompt}{answer}<|eot_id|><|start_header_id|>user<|end_header_id|>\n\n"
    )
    json_data = {"prompt": user_input, "answer": answer}

This code helps you interact with Tess. Think of it as a virtual conversation with a knowledgeable friend. Here’s what happens step-by-step:

• The code initializes a model and tokenizer, much like preparing your reference book before writing an essay.
• It sets up terminators—a special tool that helps recognize the end of a conversation, similar to a period at the end of a sentence.
• The generate_text function takes your questions and sends them to Tess, akin to passing notes in a classroom to receive insightful responses.
• Finally, a loop allows you to keep asking questions, with Tess responding as you would expect from your in-depth discussions with a highly knowledgeable colleague.

Troubleshooting Tips

If you encounter any issues while implementing the Tess language model, consider the following troubleshooting ideas:

• CUDA Errors: Ensure you have a compatible GPU set up, as this model requires GPU support to function efficiently.
• Installation Issues: Double-check your package installations. Ensure you are using the correct commands and Python environment.
• Timeout or Performance Issues: If the model takes too long to respond, try adjusting parameters like top_p, temperature, or generate_len for better performance.

For more insights, updates, or to collaborate on AI development projects, stay connected with fxis.ai.

Conclusion

With Tess, you have a powerful tool for harnessing the potential of AI. At fxis.ai, we believe that such advancements are crucial for the future of AI, as they enable more comprehensive and effective solutions. Our team is continually exploring new methodologies to push the envelope in artificial intelligence, ensuring that our clients benefit from the latest technological innovations.