MTA-Zero: Self-Improving Multimodal Deep Search via Online QA-Solver Co-training

📌 Highlights

MTA-Zero trains a multimodal deep-search Solver alongside a separate QA Generator that builds image-grounded, evidence-supported multi-hop questions online. The Solver’s rollout success rate provides feedback to both policies: it shapes question difficulty for the Generator and supplies Solver training groups. A fixed harness checks and merges QA hops, while GRPO updates the Generator and Solver independently. The paper’s main claim is that this online curriculum can adapt to the Solver’s changing abilities without constructing a multi-hop corpus beforehand.

In the reported six-benchmark comparison, the Qwen3-VL-8B agent-workflow baseline has a mean accuracy of 36.91%, while MTA-Zero reaches 51.58%. A fixed-versus-adaptive QA ablation reports averages of 44.28% and 52.40%, respectively, across four benchmarks. These are author-reported results under the paper’s setup; comparisons with prior specialist systems are not cost-matched. The approach’s practical constraints include dependence on web evidence, harness quality, and repeated multi-tool rollouts.

• The Generator and Solver are distinct policies with separate GRPO objectives.
• The QA reward favors mixed rollout outcomes near a target Solver success rate.
• The reported 8B mean accuracy rises from 36.91% to 51.58% across six benchmarks.

🎯 Introduction

The paper studies open-world multimodal deep search: an agent must ground a question in an image, decide what to search for, use external tools, and integrate web evidence into a multi-hop answer. The authors describe multimodal large language models as strong visual reasoners while identifying this tool-mediated, multi-step setting as a remaining challenge. Their target deployment-time agent is called the Solver.

The prior approach discussed is an offline data factory: tools and capable models generate candidate questions and trajectories, then verification and filtering produce training corpora. The authors argue that such corpora have fixed question distributions. As a Solver changes, examples can become trivial, unanswerable, or disconnected from its current capability gaps. Online task generation could adapt the curriculum, but multimodal search adds constraints: questions need image grounding, evidence support, entity continuity, and an independent answer path. Simply rewarding failure risks unanswerable tasks; rewarding success alone risks trivial ones.

MTA-Zero’s paper-level objective is to co-train a QA Generator and Solver so that verified question generation tracks the current Solver. It starts from image-grounded one-hop VQA seed contexts, constructs multi-hop questions online, and uses Solver rollout outcomes to favor questions near a learnable frontier. The authors report no pre-generated multi-hop corpus or additional task-specific SFT stage in their described training approach.

🔬 Methodology

At each iteration, the system starts with a seed context C = (I, q1, a1): an image, an initial question, and its answer. The trainable QA Generator extends that context by searching for information about the preceding answer and proposing a new question, answer, evidence, and source. The fixed harness filters retrieved pages into evidence chunks, checks whether each hop is anchored to and requires the current entity, and tests answer uniqueness, temporal stability, and self-contained wording. It rejects vague, generic, or non-chainable proposals. Accepted hops are merged into a running question that omits the explicit intermediate answer, and the merged question is checked for a unique intended answer and whether a direct-answer check can solve it without search.

For each accepted question, the Solver makes K independent ReAct tool-use rollouts. At each step it selects a tool and arguments or terminates with an answer, subject to the configured interaction limit. The terminal correctness of rollout j is c_ij, and the group success rate p_i is computed as follows:

The rate p_i drives two separate training signals. The Solver is updated with GRPO only for mixed groups, where some but not all rollouts are correct. The Generator receives the Goldilocks reward shown below: universal failure and universal success get zero, while the reward peaks at target success rate τ. Each policy has its own parameters, optimizer, and gradient updates; no gradients pass between policies.

In the main setting K = 8 and τ = 0.5, so the reward peaks when four of eight Solver rollouts are correct. The main experiments use web-text search for QA construction; the Solver’s listed tools include web search, image search, web reading, reverse image search, OCR, and image cropping. The QA policy’s unsuccessful proposal tokens are masked from its training loss, while accepted-hop trajectories provide training data. This combination of learned proposals and fixed acceptance checks is central to the method’s reliability claim.

📊 Experiments

The evaluation covers six multimodal deep-search benchmarks: MMSearch-Plus (MMSrch-Plus), HR-MMSearch-VL (HR-MMSrch), BrowseComp-VL (BC-VL), MMSearch-VL (MMSrch), the testmini split of VDR-Bench (VDR), and the official FVQA test split. Training seeds come from InfoSeek, the FVQA training split, InfoVQA, and OK-VQA; these provide image-grounded one-hop contexts rather than pre-generated multi-hop questions. The paper reports mean accuracy over three generation seeds and an unweighted six-benchmark average when all six scores are available. Higher accuracy is better.

The main comparison evaluates GPT-5, Claude 4.5 Sonnet, Gemini-3-Pro, and Qwen3-VL-8B-Instruct both as direct-answer models and under an agent workflow with the same framework and tool configuration used for MTA-Zero. The authors report Qwen3-VL-8B-Instruct rising from 36.91% to 51.58% average under the workflow. GPT-5 reaches 51.98% in that workflow, a 0.40-point difference from MTA-Zero’s reported average. The compared prior specialist-agent rows are reported from their original papers and are explicitly not a cost-matched comparison.

For the 8B training setup, each optimization iteration draws eight seeds, generates eight QA trajectories per seed toward five hops, and answers each retained question with eight independent Solver rollouts. The policies train independently with GRPO for 50 iterations at learning rate 10^-6 and τ = 0.5; the Solver has a ten-step interaction cap. In the fixed-versus-adaptive ablation, the four-benchmark average is 35.87% for the base Solver, 44.28% with frozen QA, and 52.40% with adaptive QA. In the target ablation, τ = 0.50 reports 52.4% overall versus 49.1% for τ = 0.25 across four benchmarks.

The analysis also evaluates checkpoint cross-play on 128 held-out seed QA pairs, generating four questions per seed and estimating each question’s Solver success rate from eight rollouts. With the Solver fixed at S50, the informative-group rate increases from 44.3% for Q0 to 56.8% for Q50. A larger-backbone comparison reports Qwen3.6-27B increasing from 46.88% to 57.70% overall on four listed benchmarks. These are the paper’s new author-reported results; external benchmark rows are contextual prior-work results rather than controlled experiments.

fig:mds-zero-overview

Caption: Overview of MTA-Zero. The QA Generator and Solver are updated online from separate GRPO objectives, while the fixed harness executes and validates QA-construction actions. The QA-side tool palette depicts the general harness interface; the main experiments enable web-text search only for QA construction.

Why It Matters: Shows the two-policy training loop and separates the learned policies from the fixed QA-construction harness.

Figure 1

Caption: Figure 1: Why co-training the QA policy matters. Left: a frozen but valid QA policy pro- duces progressively fewer informative questions as the Solver improves: the fraction of held-out QA groups with non-degenerate rollout outcomes (0 < pi < 1) falls from 55.8% at S0 to 44.3% at S50. When the QA policy is updated online alongside the Solver, matched checkpoints retain or restore this training signal. Right: per-benchmark accuracy for selected 8B agents on MMSrch-Plus, BC-VL, MMSrch, and FVQA. MTA-Zero is highlighted in teal; the MMSrch bar includes the stronger reported specialist baseline. Missing bars denote benchmarks not reported by the corresponding prior work.

Why It Matters: Combines the curriculum-mismatch evidence with selected per-benchmark comparisons; the caption’s numeric trend is also discussed in the paper’s analysis.

🔮 Conclusion

The authors conclude that verified online question generation can provide a curriculum that adapts to a changing multimodal deep-search Solver. Their reported Qwen3-VL-8B result improves the six-benchmark mean from 36.91% to 51.58% under the shared agent workflow, and a separate comparison reports improvement for Qwen3.6-27B. The cross-play and fixed-versus-adaptive analyses support the claim that updating the QA policy helps preserve informative training groups.

The practical takeaway is that task generation and task solving can be optimized together without passing gradients between the policies. The conclusion identifies richer tool environments and lower multi-tool rollout cost as promising directions. Results should be interpreted within the stated experimental setups and the checks supplied by the fixed harness.

🛠️ Future Research Improvements

The paper explicitly points to extending the harness to richer tool environments and reducing the cost of multi-tool rollouts. These are natural next investigations because the method repeatedly executes independent Solver trajectories for generated questions, and because the QA construction harness in the main experiments enables web-text search only even though its diagram depicts a broader interface. A follow-up could evaluate which additional QA-side tools improve evidence quality and whether the curriculum remains stable as the available actions change.

Further work should test transfer beyond the reported benchmark and model settings, and report how results vary with rollout count, hop depth, and the target success rate. The supplied main-body evidence tests τ = 0.50 against τ = 0.25 and reports a four-benchmark target ablation; it does not establish that one target is best across tasks or model families. Reproduction should also inspect sensitivity to retrieval failures and harness rejection behavior, since online question quality depends on both. These are proposed evaluation directions, not claims that the paper already validated them.

🏭 Potential Industry Use Scenarios

A plausible use is training an internal visual research assistant that must identify entities in user-provided images, search public web sources, and assemble multi-hop answers. An organization could begin with image-grounded question-answer seeds from a defined domain and use an evidence-checking harness to generate new practice questions as the Solver improves. The paper’s method is relevant when a team needs the training curriculum to track changing capabilities rather than remain a fixed collection of tasks.

Other possible applications include catalog or archive research, visual product investigation, and support workflows where an agent must connect an image to external product or documentation evidence. These are potential scenarios inferred from the method’s image grounding and web-search capabilities, not deployment results reported by the paper. A real implementation would need to validate source reliability, freshness, answer verification, and handling of sensitive or copyrighted material. The paper itself notes that live retrieval can surface unreliable or outdated content and that harness checks do not remove all such risks; human review and access controls would therefore matter in high-stakes use.

💬 Critical Analysis

The strongest part of the evidence is the combination of a main comparison and targeted ablations. Under the reported shared agent workflow, MTA-Zero improves the Qwen3-VL-8B six-benchmark mean; under a four-benchmark setup, adaptive QA exceeds both the base and frozen-QA conditions. The checkpoint cross-play analysis also measures whether generated questions remain informative as the Solver changes, tying the curriculum claim to an observable property rather than accuracy alone. The reward-target comparison gives evidence that, among the two tested targets, τ = 0.50 performs better than τ = 0.25 in that setup.

Several limits affect how broadly to read these results. Prior specialist-agent rows are not controlled comparisons: the authors state that their stacks and training pipelines differ. The matched commercial-model workflow is more informative for comparison, but it still reflects the paper’s particular tools, prompts, datasets, and adjudication process. The main experiment trains for 50 iterations with many independent rollouts, so operational costs and throughput matter to any reproduction. The method also relies on a fixed harness to determine whether generated QA hops are valid; the supplied evidence does not establish that the same checks generalize unchanged to other domains or sources.

A builder should verify data leakage controls, answer adjudication, retrieval quality, and question difficulty calibration in the intended setting. The paper reports checks for grounding, uniqueness, stability, and search necessity, but the main-body results do not resolve how robust the system is under noisy sources or distribution shift. The reported performance supports the proposed co-training approach on the specified evaluation, while broader reliability, cost, and deployment claims remain questions for further measurement.