Each section below summarizes one paper's reported method, experiments, and limits from its full review. The studies use different tasks and metrics; their results are not directly comparable.
arXiv:2611.07229
Spatial-UMI-Dream: Scaling World Action Models with Metric Geometry for UMI Manipulation
Spatial-UMI-Dream is a 5B World Action Model for robot manipulation that makes metric geometry explicit by jointly modeling RGB, fisheye-aligned depth, and actions from 605 hours of UMI demonstrations. It uses two-stage pretraining—world-only RGB-D adaptation followed by joint world-action learning—and evaluates transfer with the twenty-task RoboTwin-UMI benchmark. The model reaches 79.10% in-domain success, 72.15% under combined embodiment and lighting shifts, and 85.0% average success across six real-world tasks. Matched ablations attribute gains to both metric depth and UMI-specific action pretraining.
How it works
The input at timestep t contains RGB and aligned metric depth for each of two grippers. Depth is recovered from the middle and right cameras on each gripper using stereo rectification, Fast FoundationStereo disparity estimation, stereo calibration, and mapping back into the original middle-camera fisheye coordinates. The normalized depth is encoded with a Turbo color mapping and placed beside its corresponding RGB view in a fixed dual-gripper canvas. A pretrained Wan2.2-TI2V-5B VAE encodes the complete canvas as an ordinary video frame.
What the paper evaluates
The evaluation uses the twenty-task RoboTwin-UMI simulation benchmark and six real-world bimanual tasks on the Dexmate Vega1: Pick-and-Place, Bimanual Handover, Scan-and-Place, Pack-and-Close, Handover-and-Pour, and Table Wiping. RoboTwin-UMI uses Aloha-AgileX and ARX-X5 with common UMI grippers and wrist-mounted fisheye cameras. Clean Aloha measures in-domain performance, randomized-lighting Aloha tests environmental robustness, clean ARX X5 tests embodiment transfer, and randomized-lighting ARX X5 tests combined shifts. Each simulation task-setting uses 100 evaluation episodes, while each real-world task uses 20 episodes.
Limit to keep in view
The depth representation depends on calibrated multi-camera UMI hardware, stereo reconstruction, fisheye registration, and fixed depth bounds, so robustness to calibration drift or degraded stereo inputs is not established.
arXiv:2610.40607
Fight World Model: Customizable Combat through Visual Interaction Proxies
Fight World Model (FWM) separates executable combat rules from visual rendering through a deterministic visual interaction proxy. An engine resolves fighter actions, hit events, timing, and persistent health; a compiler rasterizes these outcomes into spatially and temporally aligned cues; and a pretrained video diffusion transformer renders the continuation conditioned on the proxy and appearance references. On FightWorld Corpus, FWM achieves 99.78 / 99.24 / 99.06 action accuracy for movement, attack, and skill control, 99.79 reaction accuracy, 0.15 health MAE, and 98.48 joint fidelity, while supporting damage scaling, limited-hit immunity, lifesteal, and multi-stage skills without retraining the renderer.
How it works
The engine advances executable state with equation 1, producing both the next state and event records. Engine states are aligned to output frames, projected into visible fighter attributes, and combined with active action and reaction records. The compiler generates each proxy frame as a deterministic spatial-temporal encoding. Fighter boxes and HUD bars share identity colors; command stripes represent attempted attacks; skill markers identify casting windows; brief color inversions mark reactions; and bar length represents persistent health.
What the paper evaluates
The evaluation uses two-fighter interactions in the fixed 2D view of Street Fighter II. The renderer is trained on FightWorld Corpus and compared with adapted H3-WORLD, ReactiveGWM v1, and ReactiveGWM v2. Move Accuracy, Attack Accuracy, and Skill Accuracy measure action control; Reaction measures visible hit responses; Health MAE measures normalized HUD-health error in percentage points; J requires all applicable action, entity, timing, reaction, and health checks to pass; and J_rule requires reaction and health checks to pass together.
Limit to keep in view
The main-body evaluation is restricted to two-fighter interactions in a fixed 2D Street Fighter II view, so transfer to 3D scenes, many-agent interactions, camera motion, or other game genres is not established.
arXiv:2611.01815
When Does Latent Prediction Error Support Physical Surprise Claims in Video World Models?
This paper tests when video world-model prediction error can be interpreted as physical surprise. Across IntPhys2, Physion++, Kubric, TwoRoom, and PushT evaluations, physical information is sometimes recoverable from frozen representations and incorrect actions generally raise prediction error, but native errors also respond strongly to appearance and static-content changes. In a controlled support experiment, the same trajectory ordering produces opposite impossible-versus-possible error orderings in different settings, and a transferred threshold falsely flags many lawful clips. The evidence supports treating prediction error as a measure of model discrepancy unless its physical specificity and decision performance are validated for the target conditions.
How it works
The evaluation framework separates representation evidence from score evidence. Linear probes test whether physical labels are readable from frozen features. Geometry measures appearance displacement relative to unrelated-video distances, directional alignment across scenes, and overlap with physical-pair changes.
What the paper evaluates
The central datasets and tasks are IntPhys2 for possible-versus-impossible events, Kubric for controlled support and trajectory interventions, Physion++ Mass for static-frame and appearance diagnostics, and TwoRoom and PushT for action-conditioned prediction. IntPhys2 comparisons include VideoMAE2-G and PixelSummary. DINO-WM and JEPA-WM are evaluated on PushT with a shared frozen DINOv2 encoder, varying the predictor. Uncertainty uses bootstrap intervals over the respective independent experimental units. On held-out IntPhys2 scenes, V-JEPA2-G achieves 57.9% physical classification accuracy [54.2, 61.8] and 97.7% appearance classification accuracy [95.7, 99.3].
Limit to keep in view
The controlled Kubric support experiment covers one procedural family with restricted cameras, colors, and lighting; real-world and material generalization are untested.
arXiv:2610.40503
Algebraic World Model: From Exploration to Proofs
Algebraic World Model (AWM) turns interaction with an unfamiliar environment into an explicit, executable mathematical model that can be formalized and checked in Lean 4. The system explores deterministic, partially observed environments, maintains Python hypotheses about objects and transitions, reorganizes them into typed objects and shared parameterized rules, and then proves reachability or impossibility using witnesses, invariants, and cost lower bounds. On ARC-AGI-3, AWM replayed all 183 winning trajectories across 25 games in Lean, achieved a mean score of 99.9, and used 43% of the human baseline's certified actions.
How it works
AWM models deterministic, goal-directed environments with partial observations in a POMDP setting. Observations, history, and auxiliary information are parsed into typed states w; rules specify applicability, effects, and costs; and a goal predicate identifies successful states. During Explore & Prototype, the concept book is repeatedly checked against the transition corpus using the exploration gate in equation 1. This prevents contradicted or failing hypotheses from silently remaining active while the agent continues probing.
What the paper evaluates
The main benchmark is ARC-AGI-3, consisting of 25 public games and 183 levels whose rules and goals are not revealed. AWM runs with Claude Opus 5, starts each game with an empty concept book, and evaluates score, cleared games, certified action counts, Lean replay, compactness, and generalization beyond observed trajectories. The official standard harness gives Claude Opus 5 and GPT-6 Astra no code execution, while AWM and other listed systems execute code; this distinction is part of the reported setup.
Limit to keep in view
The evaluation focuses on ARC-like environments with discrete actions, immediate observable feedback, and deterministic dynamics; hidden state, delayed feedback, and stochastic environments remain untested.
Read the full reviews
- Spatial-UMI-Dream: Scaling World Action Models with Metric Geometry for UMI Manipulation · arXiv:2611.07229
- Fight World Model: Customizable Combat through Visual Interaction Proxies · arXiv:2610.40607
- When Does Latent Prediction Error Support Physical Surprise Claims in Video World Models? · arXiv:2611.01815
- Algebraic World Model: From Exploration to Proofs · arXiv:2610.40503
Source note: findings and limitations above come from the linked full paper reviews. Follow each original arXiv paper for the authors' complete methods and results.