MRC-MMAgent: Reinforcement Learning for Multi-Role Collaboration in Multimodal Tool-Using Agents

📌 Highlights

MRC-MMAgent replaces single-role terminal-reward training with a shared solve-inspect-continue policy. The inspector evaluates whether the retrieved visual-textual evidence supports the current answer and can return control to a continuation solver.

The strongest reported search-intensive result is 62.2 average accuracy for MRC-MMAgent-7B and 68.1 for MRC-MMAgent-8B across FVQA-Test, InfoSeek, SimpleVQA, LiveVQA, and MMSearch. On the high-resolution evaluation, MRC-MMAgent-8B reaches 93.2 on V* Bench and 78.5 on HR-Bench 8K.

The central ablation supports both components: removing the role-coupled reward reduces FVQA-Test from 64.0 to 62.3 and MMSearch from 62.0 to 60.2, while single-solver and two-stage alternatives perform lower.

The main unresolved question is deployment robustness: the supplied evidence does not establish performance under tool failures, changing retrieval quality, or substantially longer interaction horizons.

• Core idea: train evidence assessment as a control decision inside the agent loop.
• Training signal: task success, valid interaction, and successful correction.
• Optimization: apply trajectory-level credit to all role-generated decisions.
• Practical caveat: reported gains are benchmark-based and use a fixed tool environment.

🎯 Introduction

Vision-language models have improved substantially on perception and visual question answering, but many questions require external evidence, reverse image search, text search, region zooming, or retrieval. Existing multimodal agents therefore train tool-augmented policies for knowledge-intensive and visually complex tasks.

The paper identifies two shortcomings in prevailing reinforcement-learning formulations. First, a single solver has no dedicated trainable mechanism for judging whether evidence supports its answer. Second, terminal-only rewards do not jointly optimize evidence assessment, control, and revision over the complete interaction trajectory.

MRC-MMAgent responds by organizing solving, inspection, and continuation as role-specific behaviors under one shared policy. The objective is to make evidence-conditioned finalization and targeted recovery part of the agent's executable behavior while avoiding additional supervised trajectories or reinforcement-learning data.

🔬 Methodology

The system uses a shared policy conditioned by three frozen role prompts. At loop index k, the initial or continuation solver produces X_k, containing reasoning, actions, observations, and a candidate answer. Equation 1 formalizes this segment, while Equation 2 selects S_1 for the initial pass and S_2 for later passes.

The inspector receives the accumulated history h'_k under S_3 and emits C_k, including a rationale and a control decision. The reference answer, candidate correctness, and final answer are excluded from h'_k, as represented by Equation 4. If the inspector requests continuation, the bridge message is appended according to Equation 5, allowing the continuation solver to act on the detected gap while retaining prior evidence.

The complete rollout contains initial solving followed by zero or more inspection and continuation stages. The role-coupled reward combines final-answer accuracy, valid tool interaction, and inspection-guided correction. Full-trajectory GRPO then assigns the resulting group-relative signal to all policy-generated decisions. Equation 10 expresses the resulting likelihood gradient over the complete set of trainable events.

📊 Experiments

The evaluation covers five knowledge-intensive multimodal search and visual-question-answering benchmarks—FVQA-Test, InfoSeek, SimpleVQA, MMSearch, and LiveVQA—and three visual grounding or high-resolution settings: V* Bench, HR-Bench 4K, and HR-Bench 8K. Training uses Qwen2.5-VL-7B-Instruct and Qwen3-VL-8B-Instruct, reverse image search, text search, and image zoom. The reported setup uses 9 NVIDIA H200 GPUs, batch size 128, learning rate 5e-6, 16K-token sequences, rollout groups of 8, and 10-turn tool interactions, with strict train-test separation.

On the five search-intensive benchmarks, MRC-MMAgent-7B reaches 62.2 average accuracy, including 64.0 on FVQA-Test, 62.0 on InfoSeek, 62.7 on SimpleVQA, 60.3 on LiveVQA, and 62.0 on MMSearch. MRC-MMAgent-8B reaches 68.1 average accuracy, with 74.2 on FVQA-Test, 65.6 on InfoSeek, 74.3 on SimpleVQA, 61.3 on LiveVQA, and 64.9 on MMSearch. The paper reports that the 8B model improves over SenseNova-MARS-8B from 67.1 to 74.2 on FVQA-Test, from 70.2 to 74.3 on SimpleVQA, and from 56.2 to 61.3 on LiveVQA.

On image reasoning, MRC-MMAgent-8B records 93.2 on V* Bench, 82.1 on HR-Bench 4K, and 78.5 on HR-Bench 8K. Relative to Qwen3-VL-8B-Instruct, the reported differences are +6.8, +3.2, and +3.9 respectively. The workflow ablation shows that Ours reaches 64.0 on FVQA-Test and 62.0 on MMSearch, compared with 58.3 and 50.9 for GRPO, 58.6 and 52.3 for ReflectAgent, 59.1 and 56.0 for two-stage RL, and 62.3 and 60.2 without the role-coupled reward.

The configuration ablations indicate that the system's Finalize and Continue behavior changes with dataset, search mode, and model scale. The role-training ablation also shows a monotonic improvement in reported overall accuracy from 51.1 in Config. 4 to 52.9 in Config. 3, 62.2 in Config. 2, and 64.0 in Config. 1, supporting joint training of solving and inspection.

fig:radar_comparison

Caption: Performance comparison of agents across different scales. Each radar chart reports results on the same five evaluation datasets.

Why It Matters: It summarizes the cross-benchmark behavior of the proposed agents and makes the consistency of the gains visible across the five search-intensive datasets.

fig:2

Caption: Overview of the proposed multi-role collaborative training framework for multimodal tasks. The Solver iteratively tackles the initial query, while the Inspector analyzes visual-textual trajectories to generate suggestions. Failed agent loops are re-executed based on historical trajectories to ensure convergence.

Why It Matters: It shows the system architecture and the control flow connecting solving, inspection, continuation, and trajectory-level learning.

fig:3

Caption: Case Study. The left panel shows a failure case where the current agent loop produces an incorrect answer. The right panel shows that after the inspector reviews the visual-textual trajectory, it detects the failure; the solver then resumes from the accumulated context and successfully corrects the answer.

Why It Matters: It provides a concrete example of why explicit inspection and continuation can correct an answer that a single solver would prematurely finalize.

fig:inspector_decision_pies

Caption: Pie chart visualizations comparing ablation results across model sizes, training approaches, and benchmarks. Orange indicates REJECT, and green indicates PASS.

Why It Matters: It visualizes how inspection decisions vary across model sizes, training approaches, and benchmark settings.

🔮 Conclusion

The paper presents a multimodal reinforcement-learning framework that internalizes trajectory assessment and continuation decisions instead of treating the agent as a single solver with terminal supervision.

Across the supplied main-body comparisons, the method improves its corresponding base models and achieves leading or competitive results on search-intensive, visual-grounding, and high-resolution benchmarks while using the original MMSearch-R1 training set.

The evidence supports the value of coupling solver and inspector behavior, but the practical conclusion should remain bounded by the evaluated backbones, tools, benchmarks, and interaction protocol.

🛠️ Future Research Improvements

A natural next step is to test the same role-coupled objective across more vision-language backbones, parameter scales, and domains so that the observed gains can be separated from model-specific behavior.

The inspection decision could be extended beyond binary Finalize and Continue to calibrated confidence, evidence coverage, contradiction detection, and explicit tool-selection recommendations. Such extensions should be evaluated with metrics that distinguish correct final answers from well-supported reasoning.

Future evaluations should measure tool failures, retrieval noise, context growth, latency, and compute cost under realistic deployment conditions. Longer-horizon studies are also needed to determine whether additional continuation improves difficult cases or mainly increases interaction overhead.

Reproducers should verify the exact reward and policy-optimization implementation from the source equations, then compare full-trajectory credit against role-selective credit under matched sampling and tool budgets.

🏭 Potential Industry Use Scenarios

A visual research assistant could use the inspector to decide whether retrieved web evidence actually supports a comparison, product identification, or fact-checking answer before responding.

In retail and catalog operations, reverse image search, text retrieval, and zoom could support product matching while continuation targets missing attributes or conflicting evidence.

For technical support and visual inspection, the solve-inspect-continue loop could require an agent to verify that an image-based diagnosis or procedure recommendation is grounded in the available visual and textual evidence.

These are deployment hypotheses rather than validated product results; the paper does not establish reliability, compliance, latency, or safety performance in production environments.

💬 Critical Analysis

The strongest aspect of MRC-MMAgent is architectural alignment between the failure mode and the training signal. The inspector is not merely appended as a textual reflection step; it controls whether the trajectory terminates or returns to tool use, and the reward evaluates the consequences of that control decision together with the solver's behavior.

The ablations provide meaningful support for the mechanism: the complete method exceeds GRPO, ReflectAgent, and two-stage RL, and removing the role-coupled reward lowers both FVQA-Test and MMSearch performance. The functional ablation further suggests that training solving and inspection together matters more than optimizing either component in isolation.

The main caveat is attribution. Because the same shared policy generates all roles and receives a trajectory-level outcome signal, improvements may reflect better global coordination, better solver behavior, better formatting, or interactions among these factors. The reported tables do not fully isolate retrieval quality, evidence correctness, tool-selection quality, and answer calibration.

For builders, the paper is a strong design blueprint for self-corrective multimodal agents, but a production implementation should independently validate failure recovery, tool reliability, long-horizon behavior, and cost under the exact deployment environment.