whereismyai
Video2World
Video2World is a research benchmark that tests whether coding agents can turn an embodied video into an interactive simulated world.
Video2World benchmarks whether coding agents can turn an embodied video into an interactive simulated world. An agent watches one robot or human video and writes the scene, its physics, and the robot's actions as code. The system checks three things: whether the geometry is right, whether the motion matches the video, and whether the robot finishes the task. It includes 222 instances in 39 task families from robot, egocentric, and third-person human videos. It evaluates frontier agents on geometric fidelity, dynamic fidelity, and functional correctness.
Video2World at a glance
- Best for: Software engineer
- Pricing: Free
- Released: October 2026
- Category: AI research tool
- Website: aetherlabsai.github.io
Key features
- Inspects video frames
- Edits scene actions
- Executes simulators
- Revises feedback
- Measures scene geometry
- Simulates objects and physics
- Controls robot actions
Use it for
- Benchmarking coding agents
- Reconstructing interactive worlds from videos
- Evaluating geometric fidelity of simulated scenes
- Testing functional task correctness
Alternatives to Video2World
- OSWorld — It enables large-scale parallel evaluation of multimodal agents across various computer environments.
- AgentIntersect World — Explore your code as a 3D city and work locally with up to four AI coding agents.
- ATAS — Maintains persistent spatial and event memory across shots.
- LingBot World — Generates physically plausible dynamic scenes with action conditioning.
- Motion2code — It gives coding agents vision tools and a frame comparison loop to prevent guesswork.
- LiveFrame — It provides verified runtime, control, and stability reports with digest-bound ledgers.
- Video Prediction Policy 2 — Achieves state-of-the-art performance on the RoboDojo-Sim leaderboard.
- Feedback Bench by Enterpret — Ranks 17 coding agents using public user feedback on 63 criteria.
- Seahaven — Provides isolated, reproducible, and stateful synthetic environments for agents.
- Dreamforge — Measurable validation in the real simulator for robot policies.
- OpenVLHarness — Lifts six multimodal model backbones by 9.7 to 12.6 points across 22 benchmarks.
- deepsense.ai — A framework to evaluate agentic systems before production usage exists.
All Video2World alternatives