whereismyai
AudioCraft
AudioCraft: Generative audio research code base that produces music, sound effects, and audio compression models from raw audio signals.
AudioCraft is a code base designed to simplify the development of generative models for audio generation and compression. It integrates three main components: MusicGen for text-to-music tasks, AudioGen for text-to-sound tasks, and EnCodec for neural audio compression. The system works by using the EnCodec neural audio codec to map raw audio waveforms into discrete tokens, which are then modeled by a single autoregressive language model. These generated tokens are decoded back into audio space to produce high-quality waveforms. The framework allows for various conditioning models, such as pretrained text encoders, to control specific audio generation outputs. This unified approach models long-term audio dependencies using a single architecture with an efficient token interleaving pattern.
AudioCraft at a glance
- Best for: Software engineer, Researcher
- Pricing: Free
- Released: August 2023
- Category: AI audio tool
- Website: audiocraft.metademolab.com
Key features
- Music generation from text inputs
- Environmental sound generation
- Neural audio compression
- Autoregressive language modeling for audio tokens
- Pretrained text encoder integration
- Efficient token interleaving pattern
- Raw waveform mapping to discrete tokens
Use it for
- Generating music from text descriptions
- Producing environmental sound effects
- Compressing raw audio signals
- Researching audio generation models
Alternatives to AudioCraft
- Soundcraft — It offers an open-source, clean-room reimplementation of Pro Tools in pure Rust.
- AuK — It handles zero-shot TTS, editing, and enhancement via natural language.
- Magenta — Builds and plays AI musical instruments with open models and real-time neural synthesis.
- FireRedAudio — A unified model handling understanding and generation with decoupled continuous representations.
- Samples — Generates local audio tracks and sound effects with Stable Audio 3.
- Blokkily — Pattern-first DAW with synchronized tracker, piano roll, step views, and MCP control
- audio.cpp — Runs modern audio models locally with high-performance, native C++ execution.
- AssemblyAI — Industry-leading speech models deliver high accuracy and low latency for voice applications.
- ESPnet — Provides reproducible recipes and hundreds of pretrained models for various speech tasks.
- Piper — Runs locally and offers fast neural text-to-speech synthesis.
- FluidAudio — Runs local audio AI models on Apple Neural Engine.
- WeNet — Achieves state-of-the-art results on public speech datasets.
All AudioCraft alternatives