EasyR1

EasyR1 is an open-source reinforcement learning training framework designed for efficient and scalable development of multi-modality language models.

EasyR1 is a training framework that functions as a clean fork of the veRL project, optimized for vision-language models. It enables efficient and scalable training using HybridEngine and vLLM's SPMD mode. The framework supports a variety of models including Llama, Qwen, and DeepSeek-R1, alongside algorithms like GRPO, DAPO, and Reinforce++. Users can train with custom text, vision-text, or multi-image-text datasets. It is built for developers and researchers who need a structured environment for reinforcement learning, offering capabilities like padding-free training and LoRA fine-tuning.

EasyR1 at a glance

Best for: Software engineer, Researcher
Runs on: open source
Pricing: Free
Released: February 2025
Category: AI coding tool
Website: github.com/hiyouga/EasyR1

Key features

Supports Llama3, Qwen2, Qwen2.5, Qwen3, and DeepSeek-R1 models
Implements GRPO, DAPO, Reinforce++, ReMax, RLOO, GSPO, and CISPO algorithms
Enables padding-free training
Supports LoRA training
Integrates with Wandb, SwanLab, Mlflow, and Tensorboard
Facilitates multi-node training via Ray
Allows checkpoint resuming

Use it for

Fine-tuning vision language models with reinforcement learning
Training reasoning models on custom image-text datasets
Running multi-node reinforcement learning training jobs

Alternatives to EasyR1

LlamaFactory — Provides a code-free CLI and Web UI for training 100+ different LLMs.
OpenRLHF — High-performance distributed architecture and unified agent-based execution
rLLM — Enables reinforcement learning training for AI agents with minimal code changes.
XTuner — High-efficiency training for 200B+ scale MoE models without expert parallelism.
ART — Enhances LLM agent performance and reliability using reinforcement learning techniques.
EasyEdit — It provides a unified framework to edit, steer, and evaluate large language models.
AirLLM — Runs massive 70B+ parameter models on small 4GB consumer GPUs.
Axolotl — It supports many training methods and model architectures via simple YAML configs.
NeMo Molt — It scales fully-async agentic RL to frontier MoE scale with a small PyTorch stack.
Agent Lightning — Train existing artificial intelligence agents with reinforcement learning and zero code changes.
LMDeploy — Accelerates inference with high-performance engines and quantization tools.
Unsloth — Runs and trains models locally with lower memory usage.

All EasyR1 alternatives