If you’ve ever wished to interact with your personal documents as if you were chatting with a knowledgeable friend, IncarnaMind is your solution! This innovative tool allows you to chat with uploaded documents (PDFs, TXTs) using Large Language Models (LLMs) like GPT. This guide will walk you through the process of getting started and troubleshooting common issues.

In a Nutshell

IncarnaMind empowers you to efficiently query your documents by leveraging a unique Sliding Window Chunking mechanism combined with an Ensemble Retriever. This enables detailed responses and minimizes factual inaccuracies while using LLMs. Whether it’s for academic purposes or just personal use, the abilities of IncarnaMind can transform how you interact with documents.

Getting Started

Let’s break down the steps to install and use IncarnaMind.

1. Installation

1.0. Prerequisites

• Python version between 3.8 and 3.11, preferably through Conda.
• API keys for at least one of the following: OpenAI, Anthropic Claude, Together.ai, or a Hugging Face token for Meta Llama models.
• Your own documents to work with.

1.1. Clone the Repository

Use the following commands to clone the IncarnaMind repository:

git clone https://github.com/junruxiong/IncarnaMind
cd IncarnaMind

1.2. Set Up the Environment

Create a Conda virtual environment:

conda create -n IncarnaMind python=3.10
conda activate IncarnaMind
pip install -r requirements.txt

Install llama-cpp specifically if you want to run quantized local LLMs:

CMAKE_ARGS=-DLLAMA_CUBLAS=on FORCE_CMAKE=1 pip install llama-cpp-python==0.1.83 --no-cache-dir

2. Usage

2.1. Upload and Process Your Files

Place your files in the data directory, ensuring they are properly named for optimal performance. To ingest all your data, run:

python docs2db.py

2.2. Start the Conversation

To initiate the interactive session, execute:

python main.py

Once the system is ready, it will prompt you for your input:

Human:

2.3. Logging

Logs are automatically generated in a file named IncarnaMind.log. You can customize logging options in configparser.ini.

Understanding the Code in IncarnaMind

Think of IncarnaMind as a wise librarian. Instead of memorizing every book (file), it categorizes them into chapters (chunks) based on topics and relevance. When you ask a question, it quickly finds which chapter contains the answer, instead of flipping through pages one by one. This not only makes the search faster but also reduces the chance of misunderstanding what you’re asking.

Troubleshooting Common Issues

• If you encounter an issue related to insufficient GPU RAM while running GGUF models, consider using the Together.ai API, which supports various LLMs including llama2-70b-chat.
• Ensure all paths and API keys are correctly set in the configparser.ini file.
• If the response from the LLM seems incorrect, verify the data ingestion step and make sure your documents are named meaningfully.

For more insights, updates, or to collaborate on AI development projects, stay connected with fxis.ai.

Additional Comments

IncarnaMind addresses several challenges, such as fixed chunk sizes in traditional RAG tools and the single-document limitation. It is adaptable and aligns perfectly with both precision and semantics. The system’s multi-document conversational features are its crown jewel, making it a potent tool for complex inquiries.

Final Thoughts

At fxis.ai, we believe that such advancements are crucial for the future of AI, as they enable more comprehensive and effective solutions. Our team is continually exploring new methodologies to push the envelope in artificial intelligence, ensuring that our clients benefit from the latest technological innovations.