LitGPT

LitGPT is Lightning AI's open-source library of 20+ LLMs written from scratch, with recipes to pretrain, finetune, evaluate and deploy them.

LitGPT is Lightning AI's Apache 2.0 library of more than 20 large language models, each implemented from scratch in readable code without abstraction layers. A command-line tool runs the main workflows: pretraining, continued pretraining, finetuning on custom data, evaluation on benchmarks such as MMLU and TruthfulQA, interactive chat, and deployment as a web server with an API. Supported models include Llama 3, Gemma 3, Phi 4, Qwen3, Mistral, Mixtral and SmolLM2. Flash Attention 2, fully sharded data parallelism, quantization down to 4 bits and parameter-efficient methods such as LoRA, QLoRA and Adapter cut memory use and let training run on anything from one GPU to more than a thousand GPUs or TPUs. Validated YAML recipes hold the training settings, and any value can be overridden from the command line.

LitGPT at a glance

Best for: Software engineer, Researcher
Runs on: CLI, open source
Pricing: Free and open source (Apache 2.0)
Released: May 2023
Category: AI coding tool
Website: github.com/Lightning-AI/litgpt

Key features

From-scratch code for 20+ LLMs, including Llama 3, Qwen3 and Phi 4
CLI workflows to pretrain, finetune, evaluate, chat and deploy
LoRA, QLoRA, Adapter and Adapter v2 finetuning
4-bit and 8-bit quantization and mixed precision to cut GPU memory
Flash Attention 2, FSDP multi-GPU training and TPU support
Validated YAML training recipes with CLI overrides
Deploy a model as a web server with an API
Export to other model weight formats

Use it for

Finetune an open model on a company's own dataset
Pretrain a small LLM from scratch for research
Benchmark a finetuned model on MMLU or TruthfulQA
Serve a finetuned model as an API for an app

Alternatives to LitGPT

LiteLLM — One OpenAI-format API for 100+ LLM providers
AirLLM — Runs massive 70B+ parameter models on small 4GB consumer GPUs.
MedicalGPT — Provides a comprehensive pipeline for full-cycle LLM training and alignment.
llm-d — Deploys production-grade LLM serving on existing hardware clusters.
WebLLM — Runs high-performance language models directly inside web browsers.
Agent Lightning — Train existing artificial intelligence agents with reinforcement learning and zero code changes.
MLC LLM — Enables native optimization and deployment of AI models across platforms.
llmware — Build private, local RAG pipelines with 300+ prepackaged models and integrated document parsing.
llmfit — Auto-detects hardware to recommend and score compatible open-source LLMs.
Soup — Fine-tune and align models like Llama-3.1-8B on a 4 GB laptop GPU.
MTPLX — It runs local models twice as fast via native MTP speculative decoding.
LightLLM — It delivers high-speed inference and serving performance for large language models.

All LitGPT alternatives