TensorRT alternatives

The 12 products closest to TensorRT in what they do and who they're for, best match first.

ncnn — No runtime dependencies and supports CPU and Vulkan GPU backends. (Code, Free)
Reflex — It is optimized for cold-start latency and single-tenant decisions. (Code, Free)
MNN — It delivers high-performance inference and training with a lightweight footprint. (Code, Free)
NVIDIA Model Optimizer — Compress deep learning models to accelerate inference performance. (Code, Free)
fal — Access 1,000+ generative models and high-performance serverless GPUs. (Code, Paid)
Flare Local Inference — Optimizes memory and bandwidth for workstations and compact systems. (Code, Free)
AirLLM — Runs massive 70B+ parameter models on small 4GB consumer GPUs. (Code, Free)
ort — Provides an ergonomic Rust API for running high-performance ONNX models. (Code, Free)
iwant — Spins up cloud GPUs and open models instantly via a single command. (Code, Free)
Dlgo — Run LLM inference quickly in pure Go without Python or external runtimes. (Code, Free)
Cube Studio — An open source, end-to-end platform for MLOps and distributed model training. (Code, Free)
LMDeploy — Accelerates inference with high-performance engines and quantization tools. (Code, Free)