Nunchaku

Nunchaku is a high-performance inference engine that accelerates 4-bit neural networks using SVDQuant.

Nunchaku is a high-performance inference engine optimized for 4-bit neural networks, implementing the SVDQuant quantization technique. It is designed for researchers and developers working with large diffusion models like FLUX.1, Qwen-Image, and SANA. The engine handles 4-bit weights and activations by migrating outliers and using low-rank components to maintain visual fidelity. It offers substantial speedups and reduces memory requirements, allowing models to run efficiently on GPUs like the NVIDIA RTX 4090 and RTX 5090. Features include multi-LoRA support, ControlNet integration, and compatibility with ComfyUI via native nodes.

Nunchaku at a glance

Best for: Software engineer, Designer
Runs on: Python, open source
Pricing: Free
Released: November 2024
Category: AI image tool
Website: github.com/nunchux-ai/nunchaku

Key features

Run 4-bit neural networks with SVDQuant
Support multi-LoRA and ControlNet
Provide ComfyUI integration
Execute asynchronous offloading
Support NVFP4 precision on NVIDIA RTX 5090
Optimize kernel fusion for low-rank branches

Use it for

Generate images using quantized diffusion models
Run FLUX and Qwen-Image models locally with low VRAM
Integrate 4-bit models into ComfyUI workflows

Alternatives to Nunchaku

Infernix — Serves massive Qwen models quickly on a single consumer GPU with advanced caching.
Kyojin — Runs 100 GB-class models in 128 GB with speculative decoding.
stable-diffusion.cpp — Runs diffusion models efficiently in pure C and C++ with no external dependencies.
SD.Next — Supports dozens of models and advanced quantization for low VRAM usage.
SANA — Provides optimized pipelines for generating high-resolution images and videos.
Mochi Diffusion — Runs Stable Diffusion and FLUX models natively on Mac hardware with full offline privacy.
Flux — Runs via API or as open weights on your own infrastructure.
KohakuMossTrainer — Fits adapter training onto cards with 6 GB of VRAM.
SpawnDev.ILGPU.ML — Runs in-browser neural network inference using 6 backends without servers
Shisei — 💃 Free CC0 mannequin pose reference library for AI image tools (Nano Banana, ChatGPT, ComfyUI, Midjourney)
Invoke — Offers an industry-leading web UI and a fully integrated unified canvas.
Dsh Sd Bridge Guard — Connects DeepSeek directly to local Stable Diffusion without manual setup.

All Nunchaku alternatives