NeMo Molt

NeMo Molt is an agentic-first and PyTorch-native reinforcement learning framework for research.

Molt is a scalable, agentic-first, and HuggingFace-native reinforcement learning framework built for research. It provides a Gymnasium-aligned API where the agent is the program and the reward is any Python written inside an environment or chat agent. The runtime uses Ray for placement and async queues, vLLM for rollout, and NVIDIA AutoModel with FSDP2 for pure PyTorch training. It supports multi-turn tools, VLM environments, and LLM-as-judge graders. The codebase consists of roughly 9.2 thousand lines of RL code designed for 1T-class mixture-of-experts models.

NeMo Molt at a glance

Best for: Software engineer, Researcher
Runs on: Linux, CLI, self-hosted, open source
Pricing: Free
Released: June 2026
Category: AI coding tool
Website: github.com/NVIDIA-NeMo/labs-molt

Key features

Gymnasium-aligned agent API
Ray-based fully-async runtime
PyTorch-native FSDP2 and NVIDIA AutoModel training
Frontier-scale MoE support with expert parallelism
Token-first contract keeping tensors aligned
Small hackable surface of about 9.2k lines of code
FlashREINFORCE critic-free single-rollout training

Use it for

Training multimodal and multi-turn AI agents
Conducting reinforcement learning research at scale
Running LLM-as-judge reward evaluations
Executing supervised fine-tuning with AutoModel

Alternatives to NeMo Molt

rLLM — Enables reinforcement learning training for AI agents with minimal code changes.
ART — Enhances LLM agent performance and reliability using reinforcement learning techniques.
RLinf — Scalable open-source framework supporting real-world robots and simulators.
Agent Lightning — Train existing artificial intelligence agents with reinforcement learning and zero code changes.
NemoClaw — Provides managed security, networking, and lifecycle operations for AI agents.
Strata — Runs large AI models locally on consumer gaming PCs without server reliance.
Stable Baselines3 — Provides reliable, standard PyTorch implementations for reinforcement learning.
OmniAgent — It evolves skills in real-time and dynamically hardens security during task execution.
Ray — It orchestrates distributed infrastructure for any AI workload at any scale.
XTuner — High-efficiency training for 200B+ scale MoE models without expert parallelism.
Rax — Exploration compiles into code that runs up to 241 times faster.
AgentGarten — Runs executable code worlds with real-time neural rendering for agents.

All NeMo Molt alternatives