Welcome to the world of Repochat, the interactive chatbot designed to engage users in meaningful conversations about GitHub repositories using a Large Language Model (LLM). In this guide, we’ll walk you through the steps to set up Repochat on your local machine for an enjoyable and efficient experience. Let’s get started!

Table of Contents

• Branches
• Installation
• Usage
• Chatbot Functionality
• Issues
• License

Branches

Repochat offers two branches with distinct functionalities:

Main Branch

The main branch runs entirely on your local machine without relying on external API calls. This ensures you have greater control over your data, making it the ideal choice for those looking for a self-contained solution.

Cloud Branch

The cloud branch depends on API calls for model inference and storage, catering to users who prefer a cloud-based solution without local setup hassles.

Installation

Follow these steps to install Repochat on your local machine:

1. Create and activate a virtual environment to isolate project dependencies:

python -m venv repochat-env
source repochat-env/bin/activate

2. Clone the Repochat repository and navigate to the project directory:

git clone https://github.com/pnkvalav/repochat.git
cd repochat

3. Install the required Python packages using pip:

pip install -r requirements.txt

4. Install the llama-cpp-python library. You can choose from several options based on hardware acceleration preferences:
• Without hardware acceleration:

pip install llama-cpp-python

• With hardware acceleration:
• For OpenBLAS:

LLAMA_BLAS=ON LLAMA_BLAS_VENDOR=OpenBLAS pip install llama-cpp-python

• For cuBLAS:

LLAMA_CUBLAS=1 pip install llama-cpp-python

• For CLBlast:

LLAMA_CLBLAST=1 pip install llama-cpp-python

• For Metal (MPS):

LLAMA_METAL=1 pip install llama-cpp-python

• For hipBLAS (ROCm support for AMD cards):

LLAMA_HIPBLAS=1 pip install llama-cpp-python

• For more details on Hardware Acceleration, refer to the official README from llama-cpp-python.
• Create a folder named models in the project directory.
• Download a Language Model from the Hugging Face Model Hub. We recommend starting with: TheBloke/CodeLlama-7B-GGUF.
• Copy the downloaded model file to the models folder.
• Set the model file location in the code_llama() function of the models.py file as follows:

def code_llama():
    callbackmanager = CallbackManager([StreamingStdOutCallbackHandler()])
    llm = LlamaCpp(
        model_path=".models/code/llama-7b.Q4_K_M.gguf",
        n_ctx=2048,
        max_tokens=200,
        n_gpu_layers=1,
        f16_kv=True,
        callback_manager=callbackmanager,
        verbose=True,
        use_mlock=True
    )
    return llm

Usage

Now that you have set up Repochat, here’s how to use it:

1. Open your terminal and run the following command:

streamlit run app.py

2. Input the GitHub repository link.
3. Repochat will fetch all the files from the repository, store them in a folder named cloned_repo, and chunk them for embedding calculations using the Sentence Transformers model, specifically sentence-transformers/all-mpnet-base-v2.
4. The embeddings are stored locally in a vector database called ChromaDB.

Chatbot Functionality

Repochat provides an engaging conversational interface where you can ask questions or provide input, and the chatbot retrieves relevant documents from the vector database. It sends your inputs, along with the fetched documents, to the language model for crafting responses. By default, it utilizes the model code_llama-7b-instruct, but you can modify this based on your machine’s speed or even experiment with the 13b quantized model. Plus, the chatbot retains memory during conversations for contextually relevant responses.

Raising Issues

If you face any problems, have suggestions, or want to report a bug, please visit the Issues section of the Repochat repository. Provide detailed information about the issue, and the developers will assist you promptly.

License

This project is licensed under the Apache License 2.0. It’s crucial to review the new license terms and conditions.

Troubleshooting Ideas

If you encounter difficulties during installation or usage, here are a few troubleshooting tips:

• Ensure all dependencies are correctly installed as per the Installation section.
• Check for compatibility issues with the Python version being used.
• Verify that the model file path is correctly specified in the models.py file.
• If the chatbot does not respond as expected, try re-running the commands and ensure the GitHub repository link is valid.
• For more insights, updates, or to collaborate on AI development projects, stay connected with fxis.ai.

At fxis.ai, we believe that such advancements are crucial for the future of AI, as they enable more comprehensive and effective solutions. Our team is continually exploring new methodologies to push the envelope in artificial intelligence, ensuring that our clients benefit from the latest technological innovations.