whereismyai › Xinference
Xinference alternatives
The 12 products closest to Xinference in what they do and who they're for, best match first.
- Lumabri — Runs large AI models across multiple computers by sharing memory and layer workloads. (Code, Free)
- LangChain4j — Integrates LLMs and vector stores directly into Java applications. (Code, Free)
- Text Embeddings Inference — High-performance serving for open-source embedding models. (Code, Free)
- ncnn — No runtime dependencies and supports CPU and Vulkan GPU backends. (Code, Free)
- LMDeploy — Accelerates inference with high-performance engines and quantization tools. (Code, Free)
- ZML — It enables Python-free inference while abstracting hardware complexities. (Code, Free)
- OpenLLM — It simplifies infrastructure management and optimizes inference speed and cost. (Code, Free)
- WebLLM — Runs high-performance language models directly inside web browsers. (Code, Free)
- oMLX — It uses paged SSD KV caching to reduce agent TTFT to under 5 seconds. (Code, Free)
- vLLM — Maximizes throughput with PagedAttention and continuous batching. (Code, Free)
- MNN — It delivers high-performance inference and training with a lightweight footprint. (Code, Free)
- OptiLLM — Increases LLM reasoning accuracy by up to 10x without model training. (Code, Free)