WorldProbe: When Video World Models Fall Apart under Long-Horizon Spatial Exploration

📌 Highlights

WorldProbe evaluates video world models as interactive, persistent environments rather than as short fixed videos. Its agent adapts camera actions online while pursuing spatial objectives, making the generated geometry part of the evaluation loop.

Across 10 models, the paper reports improved local stability in newer systems but persistent weakness in long-range memory and evolution. The central practical lesson is that a high-quality frame sequence can still fail as a coherent world.

The benchmark covers 164 test cases and four probing families, with horizons up to 180 seconds. The supplied evidence reports more than 1,600 rollouts in the abstract and main prose, while the Figure 1 overview text reports 1,201 rollouts, so the total should be checked before operational reuse.

• Core claim: local visual plausibility does not imply persistent spatial consistency.
• Strongest result: long-range memory and evolution remain weak across all evaluated models.
• Most important caveat: the benchmark depends on manually designed tasks and VLM-based scoring.

🎯 Introduction

Video world models are moving from fixed-sequence synthesis toward interactive generation conditioned on camera or action inputs. The paper argues that a useful world model must do more than render individually plausible frames: it should preserve a persistent world across revisitation, reveal unseen regions coherently, and respond to camera commands without semantic drift.

The authors identify three limitations in existing evaluation. Most benchmarks use short rollouts with limited camera motion; camera trajectories are commonly fixed before generation; and metrics emphasize local or pairwise cues that do not naturally expose contradictions over extended spatial and temporal ranges.

WorldProbe addresses this gap with closed-loop agentic probing. Compared with the listed benchmarks, it combines interactive control, high probing complexity, and a maximum length of 180 seconds, while using 164 cases.

🔬 Methodology

Each probe begins with an initial frame and a spatial objective. The goal-conditioned multimodal agent receives the task specification, a short recent-frame window, and auxiliary spatial states, then predicts a discrete camera action. That action is converted to the evaluated model's control format, applied to the next generation chunk, and fed back into the next decision.

The four task families are geometry-constrained approach, open-ended traversal, cross-view inspection, and successive region exploration. Task instances are matched to suitable initial frames from public datasets cited in the paper as Ye et al. (2026), Duan et al. (2025), and Xu et al. (2026b), followed by manual task prompts describing the intended spatial objective and camera behavior.

Long-horizon control uses SAM3.1 for memory-based, text-conditioned target tracking and a geometric gate that rejects implausibly large mask displacements. Depth-Anything3 estimates depth for each chunk, which is converted into coarse voxel geometry to estimate free-space distance and override forward actions when collision risk is detected. The system uses local geometry for collision avoidance while leaving global navigation to the agent.

The metrics deliberately separate local frame quality from long-range consistency. Visual quality averages MUSIQ, MANIQA, and Q-Align after normalization to 0–1. Content ratings cover geometric validity, object validity, and scene richness on a 0–10 scale. Memory uses reconstructed camera poses, spatially separated anchors, nearby but non-identical viewpoints, and four VLM dimensions. Evolution scores entity, scene, physical, and action–outcome causal consistency. The supplied extraction does not contain a verifiable numbered source equation, so no formula is reproduced.

• Inference is refactored into step-wise chunk generation so actions can be injected online.
• Unrecoverable rollouts terminate after persistent tracking failure or oscillatory control.
• Invalid or unsuccessful rollouts are filtered using minimum-duration criteria and coarse VLM screening.

📊 Experiments

The study evaluates 10 publicly available video world models using task-specific rollout ranges of 10–60 s, 20–120 s, 20–90 s, and 30–180 s for the four task families. The evaluation uses Qwen3.6-27B as the VLM backbone, Segment-Anything-3.1 for target tracking, and Depth-Anything-3 for local geometry reconstruction. The authors report approximately four days of evaluation on 24 A100 GPUs.

For local quality, visual scores are normalized to 0–1 and content scores remain on a 0–10 scale; more negative temporal slopes indicate faster degradation. The table shows that LingBot-World v2 has the highest content mean at 7.55 and a content slope of −0.015 points/s, while ABot-World has the most negative visual slope at −5.48 × 10−3/s. The paper's qualitative conclusion is that newer models generally maintain a higher quality floor, but content quality degrades more strongly than visual quality.

For long-range consistency, memory scores cover scene, layout, entity, and texture consistency, while evolution scores cover entity, scene, physics, and causality. AlayaWorld has the highest memory overall score at 6.53, and DreamX-World has the highest evolution overall score at 5.17. These results support the paper's claim that local rendering progress has not yielded reliable persistent world state.

The main body does not report a conventional model ablation table. Instead, the agent-effectiveness analysis shows an unconstrained DreamX rollout lasting 367.9 s and using 491 agent actions, beyond the benchmark's 180 s maximum, suggesting that the probing framework can continue when the underlying model remains usable.

• Central benchmark comparison: WorldProbe evaluates closed-loop adaptive interaction rather than only fixed trajectories.
• Local-quality result: perceptual plausibility can persist after geometry, object integrity, and scene richness have deteriorated.
• Long-range result: memory and evolution scores remain around or below the midpoint of the 0–10 scale for most models.
• No conventional ablation study is reported in the supplied main-body evidence.

fig:overview

Caption: WorldProbe overview. A goal-conditioned agent adaptively probes an evolving video world through closed-loop camera control, enabling long-horizon evaluation across four task families and four evaluation dimensions.

Why It Matters: This figure summarizes the benchmark loop: task-conditioned agent actions drive generated chunks, auxiliary perception supports control, and the resulting rollout is evaluated across four dimensions.

fig:tasks

Caption: Four scene-adaptive probing tasks. Each task induces a distinct spatial exploration pattern: approaching a constrained target, traversing an open path, inspecting a scene across viewpoints, or successively exploring new regions.

Why It Matters: The four task families operationalize different failure modes, from target completion and obstacle avoidance to long-range continuity, revisitation, and expansion into new regions.

fig:local-degradation

Caption: Local quality degrades over long-horizon interaction. Per-rollout visual and content trajectories show stronger collapse in earlier models, while more recent models generally maintain a higher quality floor. Models are ordered by release time from left to right.

Why It Matters: The figure provides qualitative evidence that local quality degradation is temporal and model-dependent, with newer systems often delaying rather than eliminating collapse.

fig:memory-cases

Caption: Representative cases for spatial memory. Long-range revisits show strong inconsistencies across spatially separated viewpoints, while short-range revisits better preserve scene structure; memory scores align with visible consistency across revisitation.

Why It Matters: The cases make the central memory failure concrete: coarse local structure can survive nearby revisits while spatially separated revisits produce incompatible geometry or scenes.

fig:evolution-cases

Caption: Representative evolution behaviors. Low-scoring cases exhibit entity and scene drift, implausible dynamics, and action-inconsistent changes, while high-scoring cases preserve substantially more coherent evolution.

Why It Matters: The examples show why frame-level plausibility is insufficient: entities can change identity, scenes can drift, physics can fail, and actions can lose causal relevance.

🔮 Conclusion

WorldProbe's central conclusion is that current video world models still fall short of maintaining a persistent world during long-horizon interaction. Recent systems improve the duration over which local visual and content quality remains usable, but spatial memory and evolution fidelity remain consistently weak.

For builders, the result argues for evaluating world models with adaptive exploration, revisitation, and extended temporal reasoning. Frame-level quality metrics should be treated as necessary but insufficient evidence of interactive world consistency.

🛠️ Future Research Improvements

A direct next step is to validate VLM-based content, memory, and evolution scores against human or expert judgments, since the supplied main-body evidence does not report such calibration. Reporting agreement, evaluator sensitivity, and failure cases would clarify how much of the measured gap reflects model behavior versus evaluator variance.

The benchmark could also benefit from controller ablations, broader task and scene coverage, and explicit comparisons between camera-following errors and world-model consistency errors. Extending beyond 180 seconds and adding richer physical actions would test whether the observed failure modes persist when interaction becomes more embodied.

The discrepancy between the abstract's more than 1,600-rollout statement and the Figure 1 overview's 1,201-rollout label should be resolved in future benchmark releases and reproduction reports.

🏭 Potential Industry Use Scenarios

WorldProbe-style testing could serve as a quality gate for interactive video products that promise persistent spaces, such as navigable virtual environments, game-world prototyping tools, or camera-controllable media generation. The memory and evolution dimensions are especially relevant when users revisit locations, inspect objects from new viewpoints, or expect actions to have coherent consequences.

The protocol could also support pre-deployment evaluation of world-model components used by embodied or simulated agents. However, the paper's results do not establish real-world robotics readiness, manipulation competence, or safe operational behavior; those claims would require additional action spaces and domain-specific validation.

💬 Critical Analysis

The paper's strongest design choice is to make the trajectory adaptive. A fixed camera path can avoid newly revealed failures or fail to test whether a model responds to generated geometry, whereas WorldProbe lets the agent redirect exploration toward targets, new regions, and revisited viewpoints. The separation between local degradation and long-range consistency is also analytically useful because it distinguishes visual collapse from semantic and spatial contradiction.

The main empirical conclusion is persuasive within the benchmark: Table 4 shows that temporal degradation varies substantially across models, while Table 5 shows that memory and evolution remain weak even when local quality is sustained. The qualitative figures reinforce that the failure is not merely a metric artifact; models can preserve coarse layout while changing texture and entity identity, or produce plausible frames with implausible scene evolution.

Several issues remain open. The cases are manually designed and limited to 164 test cases, the key long-range scores rely on VLM assessment, and the main body does not provide a conventional ablation isolating the agent, tracker, voxel module, and world model. In addition, the reported rollout-count discrepancy should be reconciled. These caveats do not negate the benchmark's diagnostic value, but they matter when comparing systems or treating its scores as deployment-grade evidence.