Soup

Soup is a command-line tool for fine-tuning and aligning LLMs locally on consumer GPUs using layer streaming.

Soup is a command-line tool designed for the entire LLM post-training workflow, including data checks, configuration generation, training, evaluation, and shipping. It uses layer streaming from CPU RAM or NVMe combined with NF4 quantization to train large models on hardware with limited GPU memory. The tool supports supervised fine-tuning along with alignment methods such as DPO, ORPO, SimPO, and KTO. It works with CUDA and Apple Silicon via MLX, supporting models like Llama-3.1-8B, Qwen4-Exp, and Whisper.

Soup at a glance

Best for: Software engineer
Runs on: CLI
Pricing: Free
Released: February 2026
Category: AI coding tool
Website: trysoup.dev

Key features

Layer streaming from CPU RAM or NVMe for low-VRAM training
Quantization of streamed base models to NF4
Automated config generation for task, quantization, and learning rate
Post-training alignment with DPO, ORPO, SimPO, and KTO
Native support for MLX on Apple Silicon and CUDA hardware
Evaluation derivation and automated save gating
Model serving with tool routes and MCP server integration
LoRA task arithmetic and checkpoint verification tools

Use it for

Fine-tuning Llama-3.1-8B models on a 4 GB laptop GPU
Aligning language models locally using DPO or SimPO without extra reference weights
Fine-tuning Whisper models for speech tasks offline
Training and evaluating models on Apple Silicon using the MLX backend
Validating and converting existing post-training configurations with a single command

Alternatives to Soup

AirLLM — Runs massive 70B+ parameter models on small 4GB consumer GPUs.
MLX LM — Runs LLMs and fine-tuning natively on Apple silicon hardware.
LMDeploy — Accelerates inference with high-performance engines and quantization tools.
llm-d — Deploys production-grade LLM serving on existing hardware clusters.
MTPLX — It runs local models twice as fast via native MTP speculative decoding.
whichllm — Ranks models using real benchmarks and hardware fit instead of parameter count.
Strata — Runs large AI models locally on consumer gaming PCs without server reliance.
LlamaFactory — Provides a code-free CLI and Web UI for training 100+ different LLMs.
OptiQ — Runs entirely on Apple Silicon without PyTorch or cloud dependence.
WebLLM — Runs high-performance language models directly inside web browsers.
vLLM — Maximizes throughput with PagedAttention and continuous batching.
Dlgo — Run LLM inference quickly in pure Go without Python or external runtimes.

All Soup alternatives