Welcome to the Hands-on LLMs Course where you will embark on a thrilling journey of building a real-time financial advisor!

This course, curated by expert instructors like Paul Iusztin, Pau Labarta Bajo, and Alexandru Razvant, is designed to provide you with practical experience in developing and deploying a financial advisor powered by large language models (LLMs).

Table of Contents

• 1. Building Blocks
• 1.1. Training Pipeline
• 1.2. Streaming Real-time Pipeline
• 1.3. Inference Pipeline
• 1.4. Financial QA Dataset
• 2. Setup External Services
• 2.1. Alpaca
• 2.2. Qdrant
• 2.3. Comet ML
• 2.4. Beam
• 2.5. AWS
• 3. Install & Usage
• 4. Lectures
• 5. License
• 6. Contributors & Teachers

1. Building Blocks

Using a three-pipeline design, this course will teach you the essential components needed to build a robust real-time financial advisor.

1.1. Training Pipeline

The training pipeline is designed to:

• Load a proprietary QA dataset.
• Fine-tune an open-source LLM using QLoRA.
• Log training experiments on the Comet ML experiment tracker.
• Log inference results on Comet ML‘s LLMOps dashboard.
• Store the best model on the Comet ML model registry.

Think of the training pipeline like a bakery. Just as a baker kneads dough, bakes it, and monitors the temperature to create the perfect loaf, this pipeline loads data (the dough), fine-tunes the model (bakes it), and tracks performance (ensures it’s golden brown). The end result? A finely-tuned LLM ready to serve!

1.2. Streaming Real-time Pipeline

This pipeline will:

• Ingest financial news from Alpaca.
• Transform news documents into embeddings in real-time using Bytewax.
• Store the embeddings into the Qdrant Vector DB.

Imagine a news reporter who not only gathers the latest headlines but also quickly summarizes the news into short, bite-sized pieces of information. This is how the streaming pipeline transforms raw data into structured chunks that LLMs can easily consume.

1.3. Inference Pipeline

The inference pipeline utilizes LangChain to:

• Download the fine-tuned model from Comet’s model registry.
• Take user questions as input.
• Query the Qdrant Vector DB for related news to enhance the prompt.
• Call the fine-tuned LLM for financial advice using the user’s query and context.
• Log responses in Comet ML.

Think of the inference pipeline like a librarian who helps readers find the right book to answer their questions. The librarian (inference pipeline) locates a book (LLM) using various guides (database queries) to provide the most accurate and helpful advice.

1.4. Financial QA Dataset

This dataset was generated using GPT-3.5 to enhance the financial knowledge of our LLM.

Understanding fine-tuning is like preparing a recipe. You start with a beginner chef (LLM) and give them a cookbook (QA dataset) filled with specialized knowledge about French cuisine (financial terms). Over time, the chef learns to create delicious dishes (provide accurate financial advice) from this augmented recipe book.

2. Setup External Services

Before commencing, you will need to set up a few external tools and services. Follow these instructions as you progress through modules:

2.1. Alpaca

Follow the documentation to create a FREE account and generate the necessary API keys.

2.2. Qdrant

Create a FREE account on Qdrant and follow the authentication guide for API keys.

2.3. Comet ML

Sign up at Comet ML and generate an API KEY and a new project following this guide.

2.4. Beam

Create a FREE account at Beam and follow their installation guide.

2.5. AWS

Sign up at AWS, create credentials, and set up the AWS CLI following this guide.

3. Install & Usage

For every module, check out its README for installation and usage instructions. Modules include:

• Q&A Dataset Generator
• Training Pipeline
• Streaming Pipeline
• Inference Pipeline

4. Lectures

To maximize your learning, clone this repository and replicate the course materials. Each module includes step-by-step video lectures and articles.

4.1. Costs

The GitHub code and video lectures are free, while some articles may require a Medium subscription.

4.2. Ask Questions

If you encounter issues, create an issue in the repository or reach out to the instructors.

4.3. Video Lectures

• Intro to the course
• Fine-tuning overview
• Hands-on fine-tuning
• Real-time text embedding
• Inference pipeline

4.4. Articles

For deeper insights into each module, check out our supporting articles.

5. License

This course is open-source and released under the MIT license. Feel free to clone or fork the project while acknowledging our work.

6. Contributors & Teachers

	Pau Labarta Bajo – Senior ML & MLOps Engineer LinkedIn
	Alexandru Razvant – Senior ML Engineer LinkedIn
	Paul Iusztin – Senior ML & MLOps Engineer LinkedIn

Troubleshooting

Should you encounter any issues, especially with Poetry while trying to locate the Beam CLI inside a virtual environment, you can follow these instructions:

export COURSE_MODULE_PATH=your-course-module-path # e.g., modules/training_pipeline
cd $COURSE_MODULE_PATH
export POETRY_ENV_PATH=$(dirname $(dirname $(poetry run which python)))
ln -s /usr/local/bin/beam $POETRY_ENV_PATH/bin/beam

For more insights, updates, or to collaborate on AI development projects, stay connected with fxis.ai.

At fxis.ai, we believe that such advancements are crucial for the future of AI, as they enable more comprehensive and effective solutions. Our team is continually exploring new methodologies to push the envelope in artificial intelligence, ensuring that our clients benefit from the latest technological innovations.