Unsloth

Unsloth: Open-source framework to run and train artificial intelligence models locally.

Unsloth is an open-source framework and desktop application that lets users run and train large language models, diffusion models, vision models, text-to-speech models, and embedding models on their own local hardware. It supports platforms like macOS, Windows, Linux, NVIDIA, AMD, Intel, and CPU setups. Users can fine-tune models faster with less memory usage using techniques like LoRA, QLoRA, reinforcement learning, and pretraining. The software connects local models to agents such as Claude Code and OpenAI Codex, and provides an OpenAI-compatible API. Unsloth also supports multi-GPU training, quantization, and data preparation from files like PDFs and CSVs.

Unsloth at a glance

Best for: Software engineer, Researcher
Runs on: macOS, Windows, Linux, Web, API, CLI, self-hosted, open source
Pricing: Free
Released: November 2023
Category: AI coding tool
Website: unsloth.ai

Key features

Run and train LLMs locally
OpenAI-compatible API
Connect agents like Claude Code and Codex
Fine-tune vision, TTS, and embedding models
Multi-GPU training support
Export to GGUF, NVFP4, and FP8 formats
Web search and RAG capabilities

Use it for

Fine-tuning large language models locally
Running local inference with text and vision models
Connecting coding agents to local models
Training models with reinforcement learning

Alternatives to Unsloth

AirLLM — Runs massive 70B+ parameter models on small 4GB consumer GPUs.
ms-swift — Supports hundreds of text and multimodal models with a full training pipeline.
Ludwig — Train and fine-tune complex AI models using a single YAML configuration file.
OpenRLHF — High-performance distributed architecture and unified agent-based execution
LMDeploy — Accelerates inference with high-performance engines and quantization tools.
GLiNER — Enables zero-shot entity extraction without labeled training data on consumer CPUs.
Axolotl — It supports many training methods and model architectures via simple YAML configs.
Strata — Runs large AI models locally on consumer gaming PCs without server reliance.
Soup — Fine-tune and align models like Llama-3.1-8B on a 4 GB laptop GPU.
Freebuff — Access multiple coding AI models for free without subscriptions.
rLLM — Enables reinforcement learning training for AI agents with minimal code changes.
RunThisLLM — Provides exact hardware and VRAM requirements for over 295 local AI models.

All Unsloth alternatives