Ollama Cloud

Ollama Cloud is a Highly Scalable Cloud-native Stack for Ollama. Help us shine by turning a into a !
**This is not a Code repository, please read Components**
Development usually kicks off on your local machine, comfy and controlled.

But moving to production? That’s a huge leap — hello, delay, inconsistency, and dependence.

Why Choose Ollama Cloud?

Ollama Cloud steps in with some solid perks:

• Consistency: If it works locally, it will work at scale with no changes to your code.
• Simplicity: Can be deployed at scale with no DevOps or SRE skills.
• A Cloud Platform: Empowers developers with self-service capabilities. What you see locally is what you get in production.

Basically, Local Is Production. Maybe we should call it LIP, lol.

Notice the use of host calls over REST APIs, which significantly speeds up communication, enhances reliability, and ensures privacy. Now you can focus on building amazing AI applications without worrying about the usual steps involved in taking them to production, such as DevOps or service dependencies. You can relax, as it is now just a mere git push away!

Components

Ollama Cloud is based on the following components:

• tau: The implementation of taubyte, a solution to build autonomous cloud computing platforms.
• ollama plugin: A plugin that allows calling ollama from WebAssembly.
• dreamland: A tool that allows running a taubyte-based cloud on your computer for local development and E2E automated testing.

Set Up Your Local Environment

First, prepare your local environment with dreamland, tau-cli, docker, and the necessary tools to build the ollama plugin.

Dreamland & Tau-CLI

If you have Node.js, just run:

npm i @taubyte/dream @taubyte/cli

If not, check dreamland and tau-cli for other installation methods.

Docker

To install Docker simply use:

curl -fsSL https://get.docker.com | sh

Build Tools

Building the plugin requires Go, but Ollama itself needs:

• cmake version 3.24 or higher
• go version 1.22 or higher
• gcc version 11.4.0 or higher

On Ubuntu Linux, install GCC & CMake with:

sudo apt install build-essential

For Go, follow the steps at Go’s official installation guide.

On macOS, you can install these using:

brew install go cmake gcc

Hardware Acceleration

If you’re unsure about the necessary development libraries for your acceleration hardware, please refer to this guide.

Build the Plugin

To build the plugin, first clone the repository:

git clone https://github.com/samyfodil/ollama.git

Then, compile the C++ backend:

cd ollama
go generate ....

Finally, build the plugin itself:

cd tau
go build . -o plugin

That’s it! You should now have a plugin executable.

Start a Local Cloud

With the plugin built, it’s time to start a local cloud. Run:

dream new multiverse

Wait until it indicates that the universe is ready.

Attach Plugin

The Cloud you’ve just started lacks LLM capabilities. To add these, load the plugin by running:

dream inject attach-plugin -p path/to/plugin

If you haven’t changed directories, it should be:

dream inject attach-plugin -p $(pwd)/plugin

Now, you’re all set to kick ass!

What’s Next?

Feel free to Create your first application to dive even deeper into what you can accomplish with Ollama Cloud.

Troubleshooting

If you experience issues setting up your environment or building the plugin, here are a few common troubleshooting steps:

• Ensure all required versions of dependencies are installed correctly.
• Check your network settings if you’re facing issues when running Docker commands.
• Consult the documentation linked above for any missed steps.

For more insights, updates, or to collaborate on AI development projects, stay connected with fxis.ai.

At fxis.ai, we believe that such advancements are crucial for the future of AI, as they enable more comprehensive and effective solutions. Our team is continually exploring new methodologies to push the envelope in artificial intelligence, ensuring that our clients benefit from the latest technological innovations.