In the ever-evolving world of software engineering, the grasp of Large Language Models (LLMs) and Generative AI (GenAI) has become crucial. This blog will guide you through the essentials of this repository, which is aimed at structuring knowledge about LLMs and GenAI in a relatable format for developers, security professionals, and operations teams. Let’s dive in!

What is This Repo About?

This repository serves as a comprehensive lesson narrative on LLMs and GenAI, created with the traditional software engineer in mind. The lessons, primarily based on the Langchain framework, cover vital aspects from development, operations, and security perspectives, ensuring you equip yourself with the necessary insights.

Lessons Overview

The lessons are cleverly categorized into three main sectors: Developer, Operations, and Security. Here’s a brief look into each:

Developer Lessons

• Calling a simple LLM using OpenAI
• Debugging in Langchain
• Chatting with OpenAI as a model
• Using prompt templates
• Utilizing Docloader to read local files
• Explaining embeddings and their calculations
• Understanding the importance of splitting and chunking
• Loading embeddings and documents in a vector database
• Implementing a Q&A chain for the RAG pattern
• Using OpenAI documentation for generating real-time information calls
• Implementing an Agent for fetching real-time information

Operations Lessons

• Token usage tracking and cost analysis
• Caching LLM calls with exact matching or embeddings
• Caching embeddings calculations locally
• Running a local LLM using Ollama
• Tracking calls and logging to a file
• Imposing output structure as JSON and implementing retries

Security Lessons

• Understanding OWASP Top 10 for LLMs
• Simple prompt injection and mitigation strategies
• Detecting prompt injection using third-party models
• Using prompts to detect project injection
• Assessing LLM responses for reliability
• Using Huggingface model for toxicity detection in LLM outputs
• Prompting LLMs for opinions on Kubernetes and Trivy vulnerabilities

History of This Repo

The structure of these lessons took shape during a GenAI hackathon hosted by Techstrong Media Ops. The lessons received further refinement during a presentation at the London DevOps Meetup group.

How Can You Help?

Collaboration makes an endeavor flourish! Here’s how you can contribute:

• Suggest topics you’d like to see added by opening a GitHub issue.
• Submit new lessons or corrections to enhance quality.
• Host your own meetup or hackathon using this repository as a base and share your experience!
• Send a thank-you tweet to @patrickdebois.

Requirements to Run This Repo

This project can be run in various ways, with a preference for using a development container.

Run It Using a Devcontainer

• This project contains a devcontainer for local execution.
• Alternatively, you can use Google Colab to run the notebooks.

Run It Locally

• Use Microsoft VSCode for demos.
• Run Python Jupyter notebooks locally.
• Utilize Poetry as your Python package manager.

poetry init 
poetry install --no-root

Configure VSCode to Use Poetry

1. Install Python 3.11 (most examples work with 3.12).
2. Set the Python version for Poetry using pyenv.
3. Get the path using pyenv which python.
4. Set the Poetry Python version with poetry env use.
5. Find and add the Poetry environment path in VSCode.
6. Add ipykernel to your dependencies.

Configure Jupyter Notebooks

• Install the Jupyter Notebook plugin for VSCode.
• Install ipykernel.

Troubleshooting Ideas

If you face issues while running the repo, consider the following troubleshooting steps:

• Ensure all dependencies, particularly Poetry and ipykernel, are correctly installed.
• Check if the Python version matches what the project requires.
• Refer to the official documentation of Langchain for version-specific guidance.

For more insights, updates, or to collaborate on AI development projects, stay connected with fxis.ai.

At fxis.ai, we believe that such advancements are crucial for the future of AI, as they enable more comprehensive and effective solutions. Our team is continually exploring new methodologies to push the envelope in artificial intelligence, ensuring that our clients benefit from the latest technological innovations.