Welcome to the world of Whisper Playground, a remarkable tool that enables you to create real-time speech-to-text applications in 99 different languages! With the powerful combination of faster-whisper, Diart, and Pyannote, you can bring your audio projects to life in no time. This guide will walk you through the setup process step by step.

Getting Started

Before we dive into the steps, make sure you have the required software installed on your device:

• Conda
• Yarn

Setup Steps

1. Clone or fork the Whisper Playground repository from GitHub.
2. Run the installation script for both backend and frontend environments by executing the command:

sh install_playground.sh

3. Review config.py to ensure the transcription device and compute type are set to match your setup. Similarly, check config.js to ensure it aligns with the backend configuration and addresses.
4. Start the backend server by navigating to the backend directory:

cd backend
python server.py

5. In a different terminal window, run the React frontend:

cd interface
yarn start

Accessing Pyannote Models

This repository utilizes models stored on the Hugging Face Hub via pyannote.audio. Please ensure you:

1. Accept the terms for the pyannote segmentation, embedding, and speaker-diarization models.
2. Install huggingface-cli and log in using your access token which can be found in your account settings.

Parameters for Configuration

While configuring your Whisper application, you can set various parameters:

• Model Size: Select from tiny to large-v2.
• Language: Specify the language to be used for transcription.
• Transcription Timeout: Set how long the application will wait before transcribing ongoing audio.
• Beam Size: Modify the number of potential transcriptions to consider.
• Transcription Method: Choose between real-time dummy setup for immediate feedback or sequential for more context-driven breakdowns.

Troubleshooting

If you encounter issues during setup, here are a few troubleshooting tips to consider:

• If you’re using MacOS and the installation fails while building the wheel for safetensors, try installing Rust using:

brew install rust

• Check if you have a stable internet connection while accessing Hugging Face models.
• Ensure you have accepted all necessary terms on your Hugging Face account.
• If you come across recognized bugs such as uncontrolled speaker swapping in sequential mode or issues with audio data timing, consider visiting the linked GitHub issues:
• Speaker Swapping Issue
• Audio Data Timing Issue

For more insights, updates, or to collaborate on AI development projects, stay connected with fxis.ai.

Final Thoughts

At fxis.ai, we believe that such advancements are crucial for the future of AI, as they enable more comprehensive and effective solutions. Our team is continually exploring new methodologies to push the envelope in artificial intelligence, ensuring that our clients benefit from the latest technological innovations.