whereismyai
Speech-to-Speech
Speech-to-Speech is an open-source, low-latency voice-agent pipeline combining VAD, STT, LLM, and TTS.
Speech-to-Speech provides a low-latency, fully modular voice-agent pipeline using VAD, STT, LLM, and TTS components. It is exposed through the core OpenAI Realtime GA event set over WebSocket and WebRTC. Every component in the pipeline is swappable, allowing users to point the LLM slot at a hosted provider, HF Inference Providers, or local servers like vLLM and llama.cpp. The software supports platforms including Apple Silicon and Linux with NVIDIA GPUs, running locally or with hosted language models. It is designed for developers building voice agents and conversation backends, such as those used in Reachy Mini robots.
Speech-to-Speech at a glance
Key features
- Voice Activity Detection using Silero VAD v5
- Speech to Text with Parakeet TDT and Whisper
- Language Model integration supporting OpenAI-compatible protocols
- Text to Speech using Qwen3-TTS and other backends
- WebSocket and WebRTC server modes
- Local execution on Apple Silicon and NVIDIA CUDA hardware
Use it for
- Build custom voice agents with open-source models
- Run fully local voice pipelines without sending audio data to third parties
- Connect browser interfaces or custom apps to a realtime speech server
- Power conversational robots with low-latency speech interactions
Alternatives to Speech-to-Speech
- AssemblyAI — Industry-leading speech models deliver high accuracy and low latency for voice applications.
- Pipecat — Builds real-time voice and multimodal agents with frame-level control.
- RealtimeTTS — Provides low-latency speech synthesis with a broad engine matrix and multiple backends.
- WeNet — Achieves state-of-the-art results on public speech datasets.
- whisper.cpp — It provides high-performance, offline speech recognition on diverse hardware.
- WhisperLive — It provides real-time speech-to-text using multiple backends like Faster Whisper and TensorRT.
- Piper — Runs locally and offers fast neural text-to-speech synthesis.
- faster-whisper — Transcribes audio faster with lower memory usage than standard implementations.
- Speaches — It acts as an Ollama equivalent for running local TTS and STT models.
- browser-whisper — Transcribes audio and video locally with zero servers involved.
- audio.cpp — Runs modern audio models locally with high-performance, native C++ execution.
- Moonshine — Runs entirely in your browser with no audio leaving your device.
All Speech-to-Speech alternatives