Welcome to the world of efficient model evaluation! If you are looking for a Rust-based tool to help you evaluate LLM models, prompts, and model parameters effortlessly, the Ollama Grid Search and AB Testing Desktop App is just what you need. This guide will walk you through the installation, usage, and troubleshooting of this powerful tool.

Purpose of the Tool

This project automates the process of selecting the best models, prompts, or inference parameters for your specific use case. It allows you to iterate over different combinations and visually inspect the results. To utilize this tool effectively, you need to have Ollama installed and serving endpoints, whether on localhost or a remote server.

Quick Example

Let’s dive right into a simple example. Imagine you want to test a basic prompt across two models using temperature values of 0.7 and 1.0. The app enables you to do just that, and you can refer to the screenshot below for a visual representation:

![Main Screenshot](.screenshotsmain.png?raw=true)

For a deeper exploration of the evaluation process using this tool, check here: In-depth Evaluation.

Installation

Getting started is easy. Follow these steps:

• Visit the releases page for the project.

Key Features

• Automatically fetches models from local or remote Ollama servers.
• Iterates over different models, prompts, and parameters to generate inferences.
• AB test various prompts on multiple models simultaneously.
• Allows multiple iterations for each parameter combination.
• Provides limited concurrency or synchronous inference calls to prevent server spam.
• Offers output of inference parameters and response metadata.
• Filters model selection by name.
• Lists experiments for download in JSON format.
• Enables inspection of experiments in readable formats.
• Configurable inference timeout and custom default parameters.

Understanding Grid Search

The concept of grid search is akin to setting out various ingredients—models, prompts, and parameters—to create a perfect dish. Imagine you have different spices (parameters) and base ingredients (models) that you can mix and match to find the flavor (optimal performance) you love. The app allows you to submit one prompt for each combination of selected parameters, helping you find that perfect blend swiftly.

AB Testing Explained

Similarly, AB testing lets you select different models to compare results for the same input. This process is like conducting a taste test among different versions of your dish using identical ingredients but varying their preparation—giving you insights into which method pleases the palate more.

![AB Testing Screenshot](.screenshotsab-animation.gif?raw=true)

Managing Experiment Logs

The app also allows you to list, inspect, or download your experiments. Think of this as keeping a detailed recipe book of all the combinations you’ve tried, along with notes on what worked and what didn’t.

![Experiment Logs](.screenshotsexperiments.png?raw=true)

Future Features

Exciting developments are on the horizon, including:

• Grading results and filtering by grade.
• Storing experiments and results in a local database.
• Facilitating the importing, exporting, and sharing of prompt lists and experiments.

Contributing to the Project

We welcome your contributions! Whether it’s fixing bugs or suggesting features, please feel free to submit a pull request or open an issue for discussion before diving into complex changes.

Development Steps

Follow these steps to develop the application:

1. Ensure Rust is installed.
2. Clone the repository with the command:

sh git clone https://github.com/dezoitool/olliama-grid-search.git

3. Navigate to the project root:

cd ollama-grid-search

4. Install the frontend dependencies using Bun:

bun install

5. Configure rust-analyzer to run Clippy for code quality.
6. Run the application in development mode:

bun tauri dev

7. Grab a cup of coffee while the app sets up!

Troubleshooting Tips

If you encounter issues, ensure that Ollama is properly installed and running. You may also want to check your Rust installation and dependencies. If you’re struggling with Llama3, please refer to the issues section here. For more insights, updates, or to collaborate on AI development projects, stay connected with fxis.ai.

Conclusion

By employing the Ollama Grid Search and AB Testing Desktop App, you’re taking significant steps toward optimizing your machine learning models intuitively and visually. At fxis.ai, we believe that such advancements are crucial for the future of AI, as they enable more comprehensive and effective solutions. Our team is continually exploring new methodologies to push the envelope in artificial intelligence, ensuring that our clients benefit from the latest technological innovations.