DR Pixel: Page-Native Deep Research with Vision-Language Models

📌 Highlights

DR PIXEL replaces the conventional OCR-then-retrieve workflow with direct retrieval and reasoning over rendered document pages. Its strongest result is 52.1% accuracy on DRPIXELBENCH for the trained Qwen3.6-27B system, compared with 40.1% for the untrained page-native model and 51.9% for GPT-5.4.

The central limitation is that the benchmark and training pipeline remain heavily model-assisted, while broader production cost, latency, and robustness on heterogeneous private documents are not fully established.

• Page-native retrieval improves untrained accuracy over OCR text-plus-figure retrieval by 5.6, 5.1, and 5.2 points across GPT-5.4, Gemini-3.5-Flash, and Qwen3.6-27B.
• The trained Qwen3.6-27B model improves from 40.1% to 52.1% accuracy on DRPIXELBENCH.
• Counterfactual credit reweighting reduces ineffective search behavior relative to vanilla SFT.
• The method transfers to IRPAPERS, M3SciQA, AirQA, and ViDoRe-V3.

🎯 Introduction

Deep-research agents answer complex questions through repeated search, reading, and synthesis over large document collections. Scientific papers are particularly challenging because decisive evidence can be distributed across prose, tables, figures, charts, and equations.

The paper argues that reducing such documents to linearized text loses visual structure and propagates OCR or parsing errors. DR PIXEL instead treats each rendered page as the retrieval and reasoning unit, preserving relationships between a figure, its table, its equation, and the surrounding explanation.

The paper's objective is to establish whether page-native deep research is both effective and efficient, then develop a training method that can improve open-weight agents without imitating every noisy step in teacher trajectories.

🔬 Methodology

The inference system follows a ReAct loop. At each turn, the VLM produces a thought and either a search query or a final answer. Offline, papers are rendered page by page, each page is encoded into a fixed-size vector, and text queries are embedded into the same space. The retriever returns top-k page images, which remain visual inputs to the agent.

The trajectory history is represented as \(\mathcal{H}_T=(\tau_1,a_1,f(a_1, \mathcal{D}),\ldots,\tau_T,a_T,f(a_T, \mathcal{D}))\). Counterfactual credit uses the frozen base model's gold-answer likelihood \(\ell(\mathcal{H}) = \log \pi_0\!\left(y^\star \mid \mathcal{H}\right)\), then computes reasoning and action credits by removing reasoning steps or comparing successive search histories.

Positive credits are converted into rank-based weights with \(w^{u}_i= \begin{cases} 0, & c^{u}_i\le 0,\\[2pt] 2\cdot\mathrm{rank}\!\left(c^{u}_i\right), & \text{otherwise.} \end{cases}\). The final token-level loss weights intermediate reasoning and action tokens separately and applies a constant \(\lambda\) to final-answer tokens.

The reported training setup fine-tunes Qwen3.6-27B with LoRA on approximately 2,500 GPT-5.4 trajectories for one epoch, using learning rate 1e-4, effective batch size 16, answer weight \(\lambda=1.5\), and four H200 GPUs.

📊 Experiments

DRPIXELBENCH contains 500 questions grounded in scientific papers from a shared corpus of 99,055 papers and approximately 1.06M rendered pages. The benchmark distinguishes Text questions from Visual questions requiring tables, figures, or equations. The evaluation also uses IRPAPERS, M3SciQA, AirQA, and ViDoRe-V3.

For the untrained comparison, GPT-5.4, Gemini-3.5-Flash, and Qwen3.6-27B use the same ReAct loop with at most 20 searches. Text-only retrieval uses five 256-word chunks; text-plus-figure retrieval adds three extracted figures when invoked; DR PIXEL retrieves three rendered pages. Accuracy is judged by GPT-5.4-mini using three independent judgments and majority voting.

DR PIXEL reaches 51.9%, 46.4%, and 40.1% accuracy for GPT-5.4, Gemini-3.5-Flash, and Qwen3.6-27B, respectively, compared with 46.3%, 41.3%, and 34.9% for text-plus-figure retrieval. The trained Qwen3.6-27B model reaches 52.1% on DRPIXELBENCH, while vanilla SFT reaches 49.0% and uses 14.0 searches per question.

On IRPAPERS, M3SciQA, and AirQA, the trained model reaches 79.4%, 64.2%, and 52.3% accuracy, compared with 65.4%, 49.8%, and 42.7% for the untrained model. On ViDoRe-V3, trained page retrieval reaches 70.8 global accuracy versus 66.7 for the untrained model and 72.6 for the page oracle.

fig:DR_Pixel

Caption: Figure 1: Overview of DR PIXEL. DR agents alternate between search, retrieval, and reasoning. Prior work operates in text space over OCR-extracted papers; DR PIXEL operates natively in visual space, with both retrieval and reasoning performed directly over rendered page images.

Why It Matters: This figure clarifies the central architectural change: replacing OCR text retrieval with direct page-image retrieval while preserving iterative agent reasoning.

fig:modality_gpt_settings

Caption: Figure 2: GPT-5.4 accuracy by evidence source under the two OCR-based settings and DR PIXEL. DR PIXEL consistently outperforms OCR-based pipelines on questions that require tabular and figure evidence.

Why It Matters: It isolates where page-native retrieval provides the clearest advantage: questions requiring tables and figures.

fig:credit_qualitative

Caption: Figure 3: An illustration of counterfactual credit reweighting (§5.1). The first search returns three off-topic pages and lowers the gold score, so the step has a negative score and weight $0$. The second retrieves the gold paper, though not the exact gold page. This increases the gold score and receives positive weight. The third retrieves the exact evidence pages and likewise receives positive weight.

Why It Matters: The example shows how the training method suppresses harmful searches while rewarding searches that improve evidence quality.

fig:corpus_scaling

Caption: Figure 4: Effect of corpus size on retrieval and final answer accuracy. Each panel varies the number of indexed documents from $20,000$ to $99,055$. DR PIXELTrained is the most robust to corpus growth, with the slowest degradation in retrieval performance.

Why It Matters: The scaling result tests whether the trained policy remains useful as the indexed corpus grows.

tab:setab-modality

Caption: Each training mixture improves the question type it matches, with Text trajectories helping text questions and visual trajectories helping multimodal ones.

Why It Matters: This ablation connects training-data modality with downstream question type and supports the emphasis on visual trajectories.

🔮 Conclusion

The paper concludes that OCR is not required for effective deep research over complex documents. Direct page retrieval preserves visual evidence, improves multimodal answer accuracy, and uses fewer retrieval calls than OCR-based alternatives.

Counterfactual credit reweighting brings a 27B open-weight model to GPT-5.4-level DRPIXELBENCH accuracy while suppressing redundant searches. The transfer results indicate that the learned query and evidence-use behavior is not limited to the benchmark's scientific-paper domain.

The practical value is highest for private or layout-rich corpora where maintaining format-specific parsers and OCR pipelines is expensive or brittle.

🛠️ Future Research Improvements

A useful next step is to evaluate larger and more heterogeneous corpora, especially scanned documents, multi-column layouts, tables spanning pages, and documents with dense mathematical notation. The current results establish robustness up to 99,055 papers, but not all real-world document conditions.

Future work should separate retrieval quality, visual reading quality, answer faithfulness, cumulative context cost, wall-clock latency, and memory use. The paper reports tool calls and token reductions, but these do not fully determine deployment cost.

Training studies should vary the number and source of teacher trajectories, compare mixed Text and Visual mixtures at equal budgets, and test whether counterfactual credit remains reliable when teacher answers or gold evidence are noisy.

🏭 Potential Industry Use Scenarios

Scientific and technical knowledge assistants could use page-native retrieval to answer questions whose evidence is distributed across paragraphs, plots, tables, and equations without maintaining separate OCR and figure-search indexes.

Legal and financial research systems are plausible deployment targets because filings, contracts, reports, and exhibits often rely on layout, tables, footnotes, and visual structure. The page-image representation also avoids requiring a custom parser for every document format.

Enterprise document search could combine page-native retrieval with access control and document-level provenance, while using resolution controls to balance visual fidelity against inference cost.

💬 Critical Analysis

The strongest evidence is the consistent untrained comparison across three model families: page-native retrieval improves both accuracy and retrieval efficiency, with the clearest gains on table- and figure-dependent questions. The trained Qwen3.6-27B results further show that visual retrieval can be paired with targeted agent training rather than treated as a fixed retrieval-only component.

The main methodological risk is benchmark dependence. Questions are generated from enriched abstracts and full-paper evidence with model assistance, then checked and human-rewritten, so the benchmark is more controlled and verifiable than naturally occurring research workloads but may encode the assumptions of its construction process.

The training method is attractive because it retains SFT while introducing step-level credit, yet its counterfactual likelihood signal depends on the frozen base model and the availability of a gold answer. Whether the same weighting remains stable for long-form answers, ambiguous evidence, or noisy labels is unresolved.

The ViDoRe-V3 results are encouraging but should be interpreted carefully: the trained system reaches 70.8 global accuracy with page retrieval versus 72.6 for the page oracle, while only 1.7% of questions require visual content. This demonstrates transfer of search behavior more directly than transfer of difficult visual reasoning.