RAGEN

RAGEN is an open-source agent reinforcement learning framework for training reasoning LLMs with multi-turn RL and diagnostics.

RAGEN is a flexible reinforcement learning framework for training reasoning LLMs with multi-turn reinforcement learning, built on the StarPO algorithm. It helps researchers and developers diagnose agent failure modes like the Echo Trap and reasoning collapse. The framework supports trajectory-level and turn-wise training, ten built-in environments, and a Gym-compatible interface for custom environments. RAGEN-2 introduces SNR-Adaptive Filtering and mutual information proxy metrics to detect template collapse. It is designed for machine learning researchers training agentic LLMs.

RAGEN at a glance

Best for: Researcher, Software engineer
Runs on: open source
Pricing: Free
Released: January 2025
Category: AI coding tool
Website: github.com/mll-lab-nu/RAGEN

Key features

StarPO framework for multi-turn trajectory-level optimization
Ten built-in environments including Sokoban and WebShop
Gym-compatible interface for custom environments
SNR-Adaptive Filtering for mitigating noisy gradient updates
Reasoning collapse diagnostics using mutual information proxy metrics
Support for PPO and GRPO algorithms

Use it for

Train reasoning LLM agents with multi-turn reinforcement learning
Diagnose and fix template collapse and reasoning failure modes
Build and test custom agent environments using a Gym interface
Apply reward-variance rollout filtering for stable gradient updates

Alternatives to RAGEN

rLLM — Enables reinforcement learning training for AI agents with minimal code changes.
ART — Enhances LLM agent performance and reliability using reinforcement learning techniques.
Stable Baselines3 — Provides reliable, standard PyTorch implementations for reinforcement learning.
Agent Lightning — Train existing artificial intelligence agents with reinforcement learning and zero code changes.
RLinf — Scalable open-source framework supporting real-world robots and simulators.
AReaL — Bridges foundation model training with modern agent-based applications.
OpenRLHF — High-performance distributed architecture and unified agent-based execution
VoltAgent — Offers open-source flexibility with built-in supervisor agent orchestration and RAG.
Seahaven — Provides isolated, reproducible, and stateful synthetic environments for agents.
NeMo Molt — It scales fully-async agentic RL to frontier MoE scale with a small PyTorch stack.
Laminar — It automatically detects agent failure patterns without manual definitions.
Marvin — Provides simple interfaces, smart defaults, and structured data extraction.

All RAGEN alternatives