sherpa-onnx

sherpa-onnx is an open source speech processing library that runs audio tasks like recognition, synthesis, and diarization locally using onnxruntime.

Sherpa-onnx provides a framework for local execution of speech-related AI models. It supports functions including speech recognition, text-to-speech, speaker diarization, voice activity detection, and speech enhancement. Developers integrate these capabilities using various programming languages such as C++, Python, JavaScript, Java, Go, and Swift. The library runs models on CPUs and NPUs across diverse operating systems and hardware platforms, including mobile, desktop, and embedded systems. It offers both streaming and non-streaming modes for real-time applications.

sherpa-onnx at a glance

Best for: Software engineer
Runs on: Android, iOS, Windows, macOS, Linux, HarmonyOS, WebAssembly, self-hosted
Pricing: Free
Released: September 2022
Category: AI audio tool
Website: github.com/k2-fsa/sherpa-onnx

Key features

Supports streaming and non-streaming speech recognition
Performs text-to-speech synthesis
Conducts speaker diarization and identification
Provides voice activity detection
Includes audio tagging and keyword spotting
Enables speech enhancement and source separation
Runs on multiple NPU architectures
Integrates with over a dozen programming languages

Use it for

Building offline speech-to-text applications
Developing local text-to-speech services
Implementing speaker identification systems
Adding voice activity detection to embedded devices
Creating real-time audio analysis tools

Alternatives to sherpa-onnx

kokoro-onnx — Runs local TTS inference with efficient ONNX runtime performance.
AssemblyAI — Industry-leading speech models deliver high accuracy and low latency for voice applications.
ESPnet — Provides reproducible recipes and hundreds of pretrained models for various speech tasks.
WeNet — Achieves state-of-the-art results on public speech datasets.
Piper — Runs locally and offers fast neural text-to-speech synthesis.
audio.cpp — Runs modern audio models locally with high-performance, native C++ execution.
Pipecat — Builds real-time voice and multimodal agents with frame-level control.
Esp32 Pie ASR — Faster than real time streaming speech recognition on a microcontroller alone
whisper.cpp — It provides high-performance, offline speech recognition on diverse hardware.
FunASR — It supports offline, streaming, and edge deployment with multiple models.
Speaches — It acts as an Ollama equivalent for running local TTS and STT models.
FluidAudio — Runs local audio AI models on Apple Neural Engine.

All sherpa-onnx alternatives