Each section below summarizes one paper's reported method, experiments, and limits from its full review. The studies use different tasks and metrics; their results are not directly comparable.
arXiv:2610.36838
MVRCap: Towards Comprehensive Video Captioning via Hierarchical Multi-Dimensional Verifiable Rewards
MVRCap treats comprehensive video captioning as a structured visual reasoning task, using supervised fine-tuning (SFT) followed by Group Relative Policy Optimization (GRPO). Its six-part caption schema covers entity attributes, actions and interactions, temporal information, camera language, scene and environment, and style and atmosphere. The training reward combines hierarchical question-answer coverage, factuality checks for object, action, and temporal errors, and a penalty for captions exceeding a target length. On VDC, CAPability, and VC4VG-Bench, the paper reports improvements over the Qwen2.5-VL-7B baseline, with the clearest reported results including VDC average accuracy/score of 49.7/2.6 and CAPability Video Avg. F1 of 37.29.
How it works
MVRCap starts with a multi-dimensional annotation paradigm. Entity Attributes and Scene and Environment describe subjects, their properties, spatial layout, and setting. Actions and Interactions and Temporal Information represent behavior, causal effects, state changes, chronological order, and scene transitions. Camera Language captures movement, focus, and depth of field; Style and Atmosphere relates visual composition to mood.
What the paper evaluates
The authors evaluate on three named benchmarks: VDC, CAPability, and VC4VG-Bench. The stated default baseline is Qwen2.5-VL-7B-Instruct. The implementation details report 2.0 FPS for RL video input, length settings Lmax = 1280 and Lcache = 256, 8 rollouts per iteration, Qwen3-30B-A3B-Instruct as judge model, and experiments on 16 NVIDIA H100 GPUs. VDC reports accuracy and score together; higher values indicate better results in the paper’s comparisons. CAPability reports F1 by dimension, including Video Avg., defined as the mean of four dynamic dimensions, and All Avg. VC4VG-Bench reports Score/% across content and necessity categories.
Limit to keep in view
The QA coverage and factuality rewards rely on a judge LLM; the supplied main-body evidence does not quantify judge reliability or agreement with human judgments.
arXiv:2611.03622
D&A: Decoupling Description and Assessment Enables VLMs as Zero-Shot Robotic Reward Models
D&A is a prompt-based approach for estimating dense robot task progress from video and natural-language instructions. It separates visual description from task assessment: a Describer first records what happened without seeing the instruction, then an Assessor defines observable success criteria and scores the description against them. The authors report that this reduces confirmation bias in zero-shot VLM reward estimates, improves results on three reward benchmarks, and supports online and offline robot learning as well as feedback-driven policy refinement. The approach avoids robotics-specific fine-tuning, but depends on the accuracy, cost, latency, and language precision of the underlying VLM.
How it works
For video o₁:T and instruction l, the goal is a reward sequence r₁:T. D&A factors the mapping into a Describer and an Assessor. The Describer narrates the observed trajectory without access to l, reducing the chance that the task expectation shapes its account. The Assessor receives l and the description, constructs an explicit natural-language criterion for what completed success looks like, and scores each described timestep against that criterion. The core mapping is shown below.
What the paper evaluates
The authors evaluate D&A against zero-shot baselines (Naive VLM, GVL, TOPReward) and post-trained models (SOLE-R1, RoboReward, Robometer). The three reward benchmarks are D&A-Bench, RBM-EVAL-OOD, and RoboRewardBenchmark. D&A-Bench contains 54 trajectories, 27 successes and 27 failures, across 14 YAM tasks, with 10 VLM-generated instruction paraphrases per task. Kendall τα is reported for D&A-Bench and RBM-EVAL-OOD, where higher is better; it measures alignment of final-step reward predictions with the ordering of optimal, suboptimal, and failed trajectories. RoboRewardBenchmark uses discrete end-of-episode progress labels and MAE, where lower is better.
Limit to keep in view
The paper says D&A inherits frontier-model latency, token costs, context-window limits, API stochasticity, and limited granular control.
arXiv:2611.07301
ArtiVidRM: Verifiable Artifact-as-Rubric Reward Modeling for Explicit Quality Articulation in Video Generation
ArtiVidRM is a video reward model that first identifies visible generation artifacts, organizes them into a structured list with severity weights, and then uses those artifacts as explicit rubrics for quality scoring. It combines supervised artifact learning with GRPO-based reinforcement learning using format, semantic-recall, preference, and length objectives. Across out-of-domain preference benchmarks, ArtiVidRM achieves the strongest or near-strongest results, while its artifact feedback improves Wan2.1 diffusion optimization and supports iterative refinement of Veo-3.1 outputs.
How it works
The data construction process begins with 26.5K prompt and video-pair triplets from VideoGen-RewardBench, containing overall and aspect-specific preference labels for Visual Quality, Motion Quality, and Text Alignment. Gemini-2.5-Pro and Qwen3-VL-32B independently inspect each pair, a consolidator merges equivalent artifact descriptions, and Qwen3-VL-32B verifies whether each candidate artifact is visibly supported. The verifier removes 26.8% of consolidated candidates, leaving 22,640 pairs and a mean of 6.11 artifacts per video.
What the paper evaluates
The evaluation covers pairwise preference prediction on GenAI-Bench, VBench-2.0, and MonetBench; multi-video ranking on T2VQA-DB, which contains 1,000 prompts with 10 videos per prompt and MOS labels from 27 human raters; pointwise scoring on T2VQA-DB and VideoFeedback2; diffusion optimization of Wan2.1-1.3B evaluated with VBench; and closed-source refinement of Veo-3.1. All preference evaluations are reported as out-of-domain with no training overlap. ArtiVidRM reaches 0.753 accuracy on GenAI-Bench without ties, 0.572 with ties on VBench-2.0, and 0.524 without ties on VBench-2.0. On MonetBench it reaches 0.606 with ties and 0.697 without ties, ranking second to VisionReward.
Limit to keep in view
The artifact labels are produced through an automated multimodal-language-model process and then spot-checked on a limited sample; the resulting 87.84% recall and 84.37% precision indicate useful but imperfect supervision.
arXiv:2611.03627
Not All Reasoning Generalizes: A Case Study on Learning Generalizable Reasoning Strategies in RLVR
This paper studies when reinforcement learning with verifiable rewards (RLVR) fails to generalize its reasoning strategy. Using synthetic non-bijective string-function inversion tasks, the authors show that RL trains Qwen2.5-7B-Instruct to use structured derivation on in-distribution tasks, but the model often switches to inefficient brute-force enumeration under harder out-of-distribution shifts. Co-training with bijective inversion tasks increases derivation usage and substantially improves OOD performance, supporting a bit-rate explanation in which outcome-only rewards implicitly favor reasoning steps that eliminate more of the solution space.
How it works
The benchmark uses 24 non-bijective functions, with 18 functions for RL training and 6 held out for evaluation. Training contains 50,000 tasks, and each evaluation set contains 1,024 randomly generated tasks. In-distribution examples use target outputs of length 4–6 and request two candidate solutions. The four OOD axes request three to six answers, use target outputs of length 6–10, combine both changes, or use Length Change functions where x and y have different lengths.
What the paper evaluates
The main evaluation covers the held-out inversion functions and four controlled OOD splits: More Answers, Longer Outputs, Longer Outputs & More Answers, and Length Change. MONO achieves high in-distribution accuracy and uses derivation in 85% of traces, but the paper reports that its accuracy remains above 80% only on the mild More Answers shift and falls to 6% on Length Change. MIXED improves OOD behavior by retaining substantially more derivation, with the largest reported advantage on Length Change. The bit-rate interventions shorten the rollout budget from 8,000 to 5,000 tokens or constrain the digit-level solution space.
Limit to keep in view
The central evidence comes from synthetic string-function inversion, so transfer to open-ended mathematics, coding, or other reasoning domains remains uncertain.
Read the full reviews
- MVRCap: Towards Comprehensive Video Captioning via Hierarchical Multi-Dimensional Verifiable Rewards · arXiv:2610.36838
- D&A: Decoupling Description and Assessment Enables VLMs as Zero-Shot Robotic Reward Models · arXiv:2611.03622
- ArtiVidRM: Verifiable Artifact-as-Rubric Reward Modeling for Explicit Quality Articulation in Video Generation · arXiv:2611.07301
- Not All Reasoning Generalizes: A Case Study on Learning Generalizable Reasoning Strategies in RLVR · arXiv:2611.03627
Source note: findings and limitations above come from the linked full paper reviews. Follow each original arXiv paper for the authors' complete methods and results.