Liger Kernel

Liger Kernel is a collection of Triton kernels that increases LLM training throughput and reduces memory usage during fine-tuning.

Liger Kernel provides a library of Triton-based kernels designed for efficient Large Language Model (LLM) training and post-training tasks. It works by replacing specific layers like RMSNorm, RoPE, SwiGLU, and CrossEntropy with optimized implementations that utilize kernel fusion, in-place replacement, and chunking. The library integrates with Hugging Face models via a one-line patching method and supports frameworks including PyTorch FSDP, DeepSpeed, and DDP. It also offers specific kernels for alignment and distillation tasks like DPO, ORPO, SimPO, and KTO, which can reduce memory consumption by up to 80%. Computations remain exact, ensuring convergence matches standard training runs.

Liger Kernel at a glance

Best for: Software engineer, Researcher
Runs on: Linux, open source
Pricing: Free
Released: August 2024
Category: AI coding tool
Website: github.com/linkedin/Liger-Kernel

Key features

One-line patching for Hugging Face models
Optimized layers for RMSNorm, RoPE, SwiGLU, and CrossEntropy
Exact computation with no approximations
Compatibility with PyTorch FSDP, DeepSpeed, and DDP
Post-training kernel support for DPO, ORPO, SimPO, KTO, and JSD
Minimal dependencies requiring only Torch and Triton

Use it for

Increasing throughput and memory efficiency for LLaMA 3-8B training
Optimizing multi-head LLM retraining with Medusa
Finetuning Vision-Language models like Qwen2-VL
Aligning models using ORPO trainers

Alternatives to Liger Kernel

AirLLM — Runs massive 70B+ parameter models on small 4GB consumer GPUs.
Soup — Fine-tune and align models like Llama-3.1-8B on a 4 GB laptop GPU.
LlamaFactory — Provides a code-free CLI and Web UI for training 100+ different LLMs.
Ludwig — Train and fine-tune complex AI models using a single YAML configuration file.
LMDeploy — Accelerates inference with high-performance engines and quantization tools.
vLLM — Maximizes throughput with PagedAttention and continuous batching.
llm-d — Deploys production-grade LLM serving on existing hardware clusters.
NVIDIA Model Optimizer — Compress deep learning models to accelerate inference performance.
LitGPT — Readable single-file LLM code with tested finetuning recipes
MLX LM — Runs LLMs and fine-tuning natively on Apple silicon hardware.
torchao — It speeds up training and inference while reducing memory usage with native PyTorch.
Enki LLM — Runs Llama-3.2 1B in pure Rust at 40 tok/s with live GPU to CPU migration.

All Liger Kernel alternatives