ncnn alternatives

The 12 products closest to ncnn in what they do and who they're for, best match first.

MNN — It delivers high-performance inference and training with a lightweight footprint. (Code, Free)
TensorRT — Speeds up inference by 36X compared to CPU-only platforms. (Code, Free)
fal — Access 1,000+ generative models and high-performance serverless GPUs. (Code, Paid)
BitNet — Runs 1-bit models on CPUs and GPUs with significantly reduced energy consumption. (Code, Free)
ort — Provides an ergonomic Rust API for running high-performance ONNX models. (Code, Free)
Xinference — Supports a wide range of model types and hardware, including multi-node setups. (Code, Free)
LMDeploy — Accelerates inference with high-performance engines and quantization tools. (Code, Free)
llm-d — Deploys production-grade LLM serving on existing hardware clusters. (Code, Free)
Strata — Runs large AI models locally on consumer gaming PCs without server reliance. (Code, Free)
AirLLM — Runs massive 70B+ parameter models on small 4GB consumer GPUs. (Code, Free)
Narwhal — It automatically hot-swaps prefill and decode roles without reloading model weights. (Code, Free)
SpawnDev.ILGPU.ML — Runs in-browser neural network inference using 6 backends without servers (Image, Free)