whereismyai
Video Prediction Policy 2
Video Prediction Policy 2 is an open source world-action model implementation for video prediction and robotic policy learning.
Video Prediction Policy 2 is a world-action model designed for zero-shot video prediction and action generation. The pipeline consists of event-level video pretraining on manipulation videos, single-step visual planning, and action learning via a mixture-of-transformers architecture. It is built for robotic researchers and developers evaluating policies on benchmarks like RoboDojo and LIBERO. The system uses CUDA and PyTorch dependencies, loading checkpoints directly from the repository. Model weights are hosted on Hugging Face and ModelScope, sharing components like Wan2.1 VAE, CLIP, UMT5, and tokenizers.
Video Prediction Policy 2 at a glance
Key features
- Event-level video pretraining on manipulation datasets
- Single-step visual planning with fixed prediction horizon
- Action learning through mixture-of-transformers architecture
- Zero-shot video prediction with input image processing
- RoboDojo joint training and closed-loop evaluation
- LIBERO, LIBERO-OOD, and LIBERO-PRO benchmark evaluation
- Checkpoint export and policy serving
Use it for
- Run zero-shot video prediction on single images or batches
- Train and evaluate policies on the RoboDojo benchmark
- Train and evaluate models on LIBERO benchmarks
- Export checkpoints and serve policies for robotics tasks
Alternatives to Video Prediction Policy 2
- LingBot World — Generates physically plausible dynamic scenes with action conditioning.
- Video2World — It evaluates coding agents on geometry, dynamics, and functionality across 222 tasks.
- Dreamforge — Measurable validation in the real simulator for robot policies.
- NanoJev — It runs parallel decisions and outputs probability distributions without output-token decoding.
- RoboGym — Simulate and control dual SO-101 robot arms directly in a web browser
- Simulusion — Compare and train multiple world models using one shared trainer and config file.
- Karotte — It runs tasks in isolated virtual machines for safety and consistency.
- Dreamscale Labs — Run frontier robot models in the cloud without local GPUs.
- Fightbench — It tests artificial intelligence models on game memory rather than pixels.
- VLMEvalKit — Evaluate large vision-language models across numerous benchmarks with a single command.
- Finetuning With Sampling — Official implementation of the Finetuning with Sampling research paper from Harvard.
- Every number in a tiny GPT — Inspects every lookup, matrix multiplication, and normalisation in a live transformer.
All Video Prediction Policy 2 alternatives