Welcome to the incredible world of Interactive LLM Powered NPCs! This open-source project revolutionizes how you interact with non-player characters (NPCs) in your favorite video games, allowing for engaging conversations using your microphone.

Getting Started

Ready to dive into this immersive experience? Here’s how to set up the Interactive LLM Powered NPCs in a user-friendly way:

Prerequisites

• Python: Ensure you have Python 3.10.6 installed. You can download it from here.
• GIT: Install GIT to manage your code. Download it from here.
• wget: This command-line utility allows you to download files conveniently.
• Microsoft Build Tools and Visual Studio: Necessary for compiling and building projects on Windows. Find more information here.
• FFmpeg: A multimedia framework for audio and video processing tasks. Follow these instructions.

Installation Steps

Follow these steps to get Interactive LLM Powered NPCs up and running:

1. Open a terminal.
2. Clone the repository:

   git clone https://github.com/AkshitIreddy/Interactive-LLM-Powered-NPCs.git

3. Navigate to the cloned repository:

   cd Interactive-LLM-Powered-NPCs

4. Create a Python virtual environment and activate it:

   python -m venv .venv
   .venv\Scripts\activate

5. Install the required dependencies:

   pip install -r requirements.txt

6. Download the necessary models from the SadTalker directory.
7. Set up your API key in apikeys.json.

How the Magic Happens

Imagine you’re holding a conversation with an NPC as if you were conversing with a friend. Here’s a delightful analogy to understand the project’s workflow:

Think of the function of Interactive LLM Powered NPCs like a chef preparing a gourmet meal. Each ingredient plays a crucial role. Your microphone captures voice (the voice of the customer), which is then transcribed (the order received). The facial recognition acts like a sous-chef identifying which meal to prepare (recognizing the character), while the LLM integration is akin to the chef crafting a unique recipe (generating a response). Pre-conversation files act like a selection of spices that add character to the meal, while the facial animation and speech generation offer the finishing touches, making the dish ready to be served (integrated into the game).

Directions for Use

Now that your project is set up, let’s ensure you can have conversations with the game’s NPCs:

1. Create a folder for your game.
2. Provide details about the game world in world.txt.
3. Follow various steps to create databases for NPC personalities and knowledge.
4. Install a voice script for character interactions.

Play the Game!

To enjoy the interactive elements, launch your game and open the main.ipynb file. Follow the prompts to talk to NPCs. Adjust all necessary settings according to your preferences.

Troubleshooting

If you encounter any issues, try the following:

• Ensure your microphone and webcam are properly connected.
• Check that you’ve activated your virtual environment before running the main program.
• Verify that all necessary dependencies are installed correctly.
• If problems persist, refer to the issues section on the GitHub repository.

For more insights, updates, or to collaborate on AI development projects, stay connected with fxis.ai.

Conclusion

Interactive LLM Powered NPCs aims to enhance gaming realism and immersion. The seamless integration of voice, recognition, and animation can turn your gameplay into an unforgettable adventure.

At fxis.ai, we believe that such advancements are crucial for the future of AI, as they enable more comprehensive and effective solutions. Our team is continually exploring new methodologies to push the envelope in artificial intelligence, ensuring that our clients benefit from the latest technological innovations.

Get Involved!

Join our community in expanding the compatibility of Interactive LLM Powered NPCs! By contributing games or enhancing existing features, become a part of shaping the future of NPC interactions in gaming.