LMDeploy alternatives

The 12 products closest to LMDeploy in what they do and who they're for, best match first.

llm-d — Deploys production-grade LLM serving on existing hardware clusters. (Code, Free)
AirLLM — Runs massive 70B+ parameter models on small 4GB consumer GPUs. (Code, Free)
vLLM — Maximizes throughput with PagedAttention and continuous batching. (Code, Free)
LightLLM — It delivers high-speed inference and serving performance for large language models. (Code, Free)
MLC LLM — Enables native optimization and deployment of AI models across platforms. (Code, Free)
FastDeploy — Provides high-performance production deployment for LLMs and VLMs across multiple hardware platforms. (Code, Free)
WebLLM — Runs high-performance language models directly inside web browsers. (Code, Free)
MLX LM — Runs LLMs and fine-tuning natively on Apple silicon hardware. (Code, Free)
fal — Access 1,000+ generative models and high-performance serverless GPUs. (Code, Paid)
llmware — Build private, local RAG pipelines with 300+ prepackaged models and integrated document parsing. (Code, Free)
Distributed Llama — Splits model memory and computation across multiple connected home devices. (Code, Free)
LMCache — It stores, compresses, searches, and reuses KV caches across clusters. (Code, Free)