The internet has become a treasure trove of video content, especially with platforms like TikTok and YouTube leading the charge. With so much knowledge packed into videos, how do we sift through millions to find specific snippets of information? This is where the AI Video Search Engine (AVSE) shines. Designed to index video content just like Google indexes text, AVSE makes pinpointing answers within videos easier and faster.

Why AVSE Matters

As short-form content becomes increasingly popular, the necessity of a robust video search engine has never been clearer. While Google has created a text-based indexing marvel, the vast troves of knowledge contained in video formats remain mostly inaccessible and challenging to navigate. AVSE aims to bridge this gap and facilitate easier knowledge discovery through video indexing.

Tech Stack Behind AVSE

Building a functional video search engine requires a solid tech stack. Here’s what powers AVSE:

• Supabase (PostgreSQL, PG_Vector, Auth)
• Hasura (GraphQL layer, permissions)
• Fly (Hosting of Hasura)
• JigsawStack (Summary AI, Chat AI)
• Vercel (Next.js hosting, Serverless functions)

How AVSE Works

The operation of AVSE can be broken down into several key processes, which I’ll explain using an analogy of a librarian:

• Storing of Videos: Imagine a librarian watching every single video and transcribing them verbatim. This is akin to how AVSE extracts transcription from YouTube videos, chunking and timestamping it using Hugging Face‘s tools, storing it in a PostgreSQL database with pg_vector extension for rapid indexing.
• Searching: Now, when you ask the librarian a question, they don’t just scan through books; they use a special system to look for keywords that are semantically related to your query. AVSE employs vector cosine search to efficiently sift through the database, returning relevant video snippets that match your question. Each result is associated back to specific timestamps, enabling playback of the relevant video clips.
• Summary & Chat: If you wanted a quick overview, the librarian would highlight key points using their notes. Similarly, AVSE sends the transcription to JigsawStack API for a concise summary in both point form and text, facilitating chat sessions to manage questions and related video content.

Hosting AVSE Yourself

For those eager to take the plunge into hosting AVSE, here’s what you need to know:

• A paid account on Supabase and Fly.io is essential for indexing a significant volume of videos.
• The adminconfigfly.toml file holds the necessary configuration to deploy Hasura to Fly.
• Use the migration dump adminmigration to recreate the schema through Hasura CLI.
• Update the migration folder using the command: hasura init migration --endpoint hasuraurl.fly.app --admin-secret admin_secret.
• The script adminindexChannelVideos.ts helps index large numbers of videos locally from YouTube channels.
• The .env.example file contains the keys essential for running the project.

Troubleshooting Ideas

If you encounter any issues while navigating or setting up AVSE, here are a few tips:

• Check your network connection to ensure stable communication with Supabase and Fly.io.
• Consult the log files for any errors or discrepancies during video indexing.
• If video clips aren’t returning correctly, double-check your indexing scripts for any misconfigurations.

For more insights, updates, or to collaborate on AI development projects, stay connected with fxis.ai.

FAQs

• Doesn’t YouTube do this? Not quite! YouTube indexes video metadata but does not search transcribed audio, leaving a wealth of audio content unindexed.
• How will this stack handle millions of videos? The current setup is capable of handling millions, though scaling to billions would require additional resources and cost considerations.

What’s Next for AVSE?

• Integrating TikTok as an additional video source.
• Implementing the fast audio transcription service from Replicate.
• Significantly improving query performance.
• Creating a page for users to view all active chat sessions for better usability.

At fxis.ai, we believe that such advancements are crucial for the future of AI, as they enable more comprehensive and effective solutions. Our team is continually exploring new methodologies to push the envelope in artificial intelligence, ensuring that our clients benefit from the latest technological innovations.