Welcome to the exciting world of the Medicine LLM! In this article, we will walk you through everything you need to know to get started with the Medicine LLM (Large Language Model) created by AdaptLLM. With the introduction of the GGUF format, utilizing these models has never been easier. Let’s dive in!

What is the Medicine LLM?

The Medicine LLM is a domain-specific model designed to enhance your applications in the medical field. By harnessing advanced techniques like quantization and continued pre-training on specialized datasets, this model ensures that you not only gain knowledge about medicine but also receive accurate and context-aware outputs.

How to Download GGUF Files

To make the most of the Medicine LLM, you first need to download the GGUF files. Here are a few user-friendly methods:

• Using text-generation-webui: Under Download Model, enter the model repo: TheBloke/medicine-LLM-GGUF, followed by a specific filename to download (e.g., medicine-llm.Q4_K_M.gguf).
• Command Line: For a faster method, utilize the huggingface-hub Python library by running:

  pip3 install huggingface-hub

  Then, download your desired model file with:

  huggingface-cli download TheBloke/medicine-LLM-GGUF medicine-llm.Q4_K_M.gguf --local-dir . --local-dir-use-symlinks False

Running the Model

Once you’ve downloaded the model files, it’s time to run them! Below is a quick overview of how to run the model using various methods:

Using llama.cpp

Your command could look something like this:

./main -ngl 35 -m medicine-llm.Q4_K_M.gguf --color -c 2048 --temp 0.7 --repeat_penalty 1.1 -n -1 -p ### User Input: 

Make sure to adjust parameters according to your needs—such as the number of GPU layers to offload.

Using Python Code

For Python users, here’s a simple example to load and utilize the model:

from llama_cpp import Llama

llm = Llama(

    model_path=".medicine-llm.Q4_K_M.gguf",

    n_ctx=2048,

    n_threads=8,

    n_gpu_layers=35

)
output = llm(

    prompt="### User Input: ",

    max_tokens=512,

    stop=["