Rapid-MLX

Rapid-MLX is an open-source LLM inference server and Mac app for Apple Silicon that runs models locally.

Rapid-MLX is a high-performance, OpenAI- and Anthropic-compatible LLM inference server and Mac app designed for Apple Silicon. It allows users to run models locally on their Mac without accounts or cloud inference, keeping all prompts private on the machine. The engine supports text, vision, audio, and video models, featuring continuous batching to handle multiple callers simultaneously. It is built for developers and power users who want to host local models for coding agents, applications, and network clients. Supported capabilities include tool-call parsing, chain-of-thought separation, and prompt caching.

Rapid-MLX at a glance

Best for: Software engineer
Runs on: macOS, Apple Silicon, CLI, Web
Pricing: Free and open source (Apache 2.0)
Released: February 2026
Category: AI coding tool
Website: rapidmlx.com

Key features

OpenAI- and Anthropic-compatible inference server
Continuous batching for concurrent requests
Prompt caching with KV trimming
Tool-call parser modules with automatic recovery
Chain-of-thought separation for reasoning models
Support for vision, audio, and video models
Headless server mode with optional authentication

Use it for

Run local LLMs for coding agents and editors
Serve local AI models across a local network
Execute local speech-to-text and voice cloning
Generate video locally using an async API

Alternatives to Rapid-MLX

MTPLX — It runs local models twice as fast via native MTP speculative decoding.
oMLX — It uses paged SSD KV caching to reduce agent TTFT to under 5 seconds.
MLX LM — Runs LLMs and fine-tuning natively on Apple silicon hardware.
Strata Mlx — Runs 125B MoE models on Apple Silicon by streaming experts directly from disk.
MLX Audio — It enables fast audio model inference directly on Apple M-series chips.
LiteLLM — One OpenAI-format API for 100+ LLM providers
llm-d — Deploys production-grade LLM serving on existing hardware clusters.
Laya Mlx — Provides local MLX inference with no PyTorch or cloud API dependency.
LMDeploy — Accelerates inference with high-performance engines and quantization tools.
MLC LLM — Enables native optimization and deployment of AI models across platforms.
AirLLM — Runs massive 70B+ parameter models on small 4GB consumer GPUs.
WebLLM — Runs high-performance language models directly inside web browsers.

All Rapid-MLX alternatives