Mooncake alternatives

The 12 products closest to Mooncake in what they do and who they're for, best match first.

llm-d — Deploys production-grade LLM serving on existing hardware clusters. (Code, Free)
LMCache — It stores, compresses, searches, and reuses KV caches across clusters. (Code, Free)
LMDeploy — Accelerates inference with high-performance engines and quantization tools. (Code, Free)
AirLLM — Runs massive 70B+ parameter models on small 4GB consumer GPUs. (Code, Free)
GPUStack — Unifies AI model serving and GPU compute management across heterogeneous hardware. (Code, Free)
Soup — Fine-tune and align models like Llama-3.1-8B on a 4 GB laptop GPU. (Code, Free)
GPTCache — It decreases expenses and enhances performance by caching LLM query results. (Code, Free)
SGLang — It delivers high-throughput, low-latency serving for large-scale deployments. (Code, Free)
fal — Access 1,000+ generative models and high-performance serverless GPUs. (Code, Paid)
TokenSpeed — Optimized performance for agentic workloads and high-throughput LLM serving. (Code, Free)
oMLX — It uses paged SSD KV caching to reduce agent TTFT to under 5 seconds. (Code, Free)
Narwhal — It automatically hot-swaps prefill and decode roles without reloading model weights. (Code, Free)