whereismyai
WhisperLiveKit
WhisperLiveKit is a self-hosted speech-to-text pipeline that provides real-time transcription, translation, and speaker diarization.
WhisperLiveKit is an ultra-low-latency, self-hosted speech-to-text pipeline designed for real-time transcription, translation, and speaker diarization. It uses state-of-the-art simultaneous speech research for intelligent buffering and incremental processing rather than handling complete utterances. The backend supports multiple concurrent users and includes Voice Activity Detection to reduce overhead when no voice is detected. It supports backends and models such as Simul-Whisper, WhisperStreaming, Sortformer, Qwen3-ASR, Voxtral, and FunASR SenseVoiceSmall. Users can configure sessions via WebSockets with parameters for language, target language, context, mode, and tokens, alongside various optional hardware and software dependencies.
WhisperLiveKit at a glance
- Best for: Software engineer, Writer
- Runs on: Web, macOS, API, self-hosted, open source
- Pricing: Free
- Released: December 2024
- Category: AI audio tool
- Website: github.com/QuentinFuxa/WhisperLiveKit
Key features
- Ultra-low latency transcription using Simul-Whisper and WhisperStreaming
- Simultaneous translation from and to 200 languages
- Real-time speaker diarization using Streaming Sortformer
- Voice Activity Detection for reduced overhead
- Per-session WebSocket query parameters for configuration
- Support for multiple concurrent users
- Compatibility with various optional GPU and CPU backends
Use it for
- Capture audio from web pages
- Perform real-time speech-to-text transcription
- Translate spoken audio across multiple languages simultaneously
- Conduct real-time speaker diarization
Alternatives to WhisperLiveKit
- WhisperX — It enables 70x faster, batched transcription with high-accuracy alignment.
- Whisper Diarization — Combines OpenAI Whisper with voice activity detection and speaker embedding.
- Whisper ASR Webservice — Provides a REST API for multiple ASR engines including WhisperX and Faster Whisper.
- browser-whisper — Transcribes audio and video locally with zero servers involved.
- faster-whisper — Transcribes audio faster with lower memory usage than standard implementations.
- Faster Whisper GUI — Provides a visual interface for faster-whisper, whisperX, and Demucs models.
- whisper.cpp — It provides high-performance, offline speech recognition on diverse hardware.
- Whisper Timestamped — Generates accurate word-level timestamps without extra neural networks.
- LiveKit Agents — Connects AI models to WebRTC rooms with reliable turn detection and streaming pipelines.
- Esp32 Pie ASR — Faster than real time streaming speech recognition on a microcontroller alone
- Speech-to-Speech — Fully modular, swappable components with local or hosted execution options.
- talkflow — Transcribes speech entirely on your device with no word limits or cloud accounts
All WhisperLiveKit alternatives