Stav Cohen, Ron Bitton, Ben Nassian
Technion – Israel Institute of Technology, Cornell Tech, Intuit
Website | YouTube Video | ArXiv Paper

Contents

• Overview
• Abstract
• Install
• Running the code
• RAG-based Worm
• FlowSteering Worm
• Simulating a GenAI LLaVa Ecosystem

Overview

We’ve unleashed a digital hurricane! Our work explores how a computer worm can infiltrate GenAI-powered applications, specifically targeting email assistants. We’ve demonstrated it through various layers of complexity, testing against popular models like Gemini Pro, ChatGPT 4.0, and LLaVA.

Here’s what our worm can do:

• Exfiltrating personal data
• Spamming

Take a visual look with the following illustrative examples:

Exfiltrating personal data
Spamming

Abstract

In a world rapidly embracing Generative AI (GenAI), there is an increasing interconnectivity among these technologies, making them targets for malicious intent. This paper presents Morris II, a groundbreaking worm designed to exploit GenAI systems through self-replicating prompts. These prompts push the boundaries of what a worm can do by not only replicating but also self-propagating throughout the interconnected landscape of GenAI applications.

Install

To get started with the RAG-based Worm, follow these steps:

1. Clone the repository and navigate to the multimodal injection folder:

git clone https://github.com/StavC/ComPromptMized.git
cd ComPromptMized

2. Create a Conda environment for LLaVa and install necessary packages:

conda create -n ComPromptMized python=3.10 -y
conda activate ComPromptMized
pip install --upgrade pip
pip install -r requirements.txt
cd FlowSteering
cd llava
pip install -e .

3. Download LLaVa weights from the LLaVA repository and save them in the ComPromptMized/FlowSteering/llava/llava_weights directory. We recommend using the LLaVA-7B weights.

Running the code

As our code is particularly user-friendly, we’ve transformed it into a Jupyter notebook, enhancing readability and simplifying testing. Below are the steps for running the code successfully.

RAG-based Worm

Inside the RAG-based Worm folder, you’ll discover RagPipeline.ipynb, which illustrates how a self-replicating text prompt can poison a RAG-based email application:

1. Configure the following details in the notebook:
• Path for emails: Set the CSV path (e.g., RagEmailsCsv_dir = 'RAG-based Worm/RAG Emails/Emails.csv') for your email data.
• VectorStore Config: Define the path to save/load your VectorStore database.
• API Keys: Input your OpenAI and Google API keys for functionality.
• Choose LLM: Set LLM_Name to “ChatGPT” or “GeminiPro”.

FlowSteering Worm

Our experiments were conducted with LLaVa on a single NVIDIA Quadro RTX 6000 24GB GPU. In this folder, you will find ImgPerturbation.ipynb, which details the image perturbation process:

1. Adjust your LLaVa parameters and specify the directory for the LLaVa weights.
2. Use the functions provided to manipulate images, steering the flow of GenAI applications.

Simulating a GenAI LLaVa Ecosystem

To execute and simulate a comprehensive evaluation with different clients and an email server, please refer to the Application Code readme.

Troubleshooting

While running any novel applications can present some challenges, here are some troubleshooting steps to consider:

• If the environment setup fails, ensure that all package dependencies are correctly installed.
• For errors related to API keys, double-check your configuration and ensure you’ve included valid keys.
• If outcomes are not as expected, review the configurations in your Jupyter notebook step-by-step.
• If you experience issues with loading images or the perturbation is not producing results, verify the image paths and formats.

For more insights, updates, or to collaborate on AI development projects, stay connected with fxis.ai.

Conclusion

At fxis.ai, we believe that such advancements are crucial for the future of AI, as they enable more comprehensive and effective solutions. Our team is continually exploring new methodologies to push the envelope in artificial intelligence, ensuring that our clients benefit from the latest technological innovations.