RVR: Retrieve-Verify-Retrieve for Comprehensive Question Answering

Abstract: Comprehensively retrieving diverse documents is crucial to address queries that admit a wide range of valid answers. We introduce retrieve-verify-retrieve (RVR), a multi-round retrieval framework designed to maximize answer coverage. Initially, a retriever takes the original query and returns a candidate document set, followed by a verifier that identifies a high-quality subset. For subsequent rounds, the query is augmented with previously verified documents to uncover answers that are not yet covered in previous rounds. RVR is effective even with off-the-shelf retrievers, and fine-tuning retrievers for our inference procedure brings further gains. Our method outperforms baselines, including agentic search approaches, achieving at least 10% relative and 3% absolute gain in complete recall percentage on a multi-answer retrieval dataset (QAMPARI). We also see consistent gains on two out-of-domain datasets (QUEST and WebQuestionsSP) across different base retrievers. Our work presents a promising iterative approach for comprehensive answer recall leveraging a verifier and adapting retrievers to a new inference scenario.

• The paper presents the RVR framework for iterative evidence retrieval using LLM-based verification, significantly improving answer coverage in multi-answer QA.
• Experimental results show at least a 10% relative gain in complete recall and robust performance across various retriever architectures and datasets.
• RVR balances multiple rounds of retrieval and verification to reduce redundancy and inference costs while ensuring comprehensive evidence assembly.

Retrieve-Verify-Retrieve (RVR): Iterative Retrieval for Multi-Answer Question Answering

Problem Formulation and Motivation

Comprehensive question answering requires retrieval methods that maximize not only the accuracy but also the coverage of valid answers, particularly when queries admit a broad set of correct responses. Traditional dense retrievers optimized for relevance are insufficient under this regime, as they focus on top-k recall but frequently miss long-tail or less prominent answers, leading to incomplete outputs. Existing agentic search approaches, which explore iterative search and query reformulation, are predominantly designed for multi-hop reasoning and not tailored for full answer coverage per query. There exists a need for a workflow that directly targets comprehensive retrieval: efficiently assembling all valid supporting evidence for open-domain, multi-answer queries.

The RVR Framework

The RVR framework operationalizes retrieval as a multi-stage iterative loop, with explicit conditioning on previously verified evidence. The core components are:

• Initial Retriever ($f_{i}$): Encodes the user query and retrieves a ranked set of candidate documents.
• Verifier ($g$): A binary LLM-based verifier that filters candidates, identifying a high-quality, relevant subset.
• Subsequent Retriever ($f_{r}$): Consumes the concatenated query and previously verified evidence to retrieve new, complementary documents in subsequent rounds.

At each iteration, the retrieved and verified documents are accumulated, with later rounds conditioned explicitly on previously uncovered, high-value context. The output is a merged, deduplicated set prioritized for both relevance and diversity of answers.

The training protocol involves contrastive learning with in-batch negatives and sampled hard negatives, with $f_{r}$ trained specifically on the incremental retrieval objective (finding missing evidence given earlier context).

Experimental Evaluation

Main Results on QAMPARI:

The framework demonstrates statistically significant improvements over both strong baselines and state-of-the-art agentic methods. The RVR system achieves at least a 10% relative and 3% absolute gain in MRecall@100 (complete recall percentage) on the QAMPARI dataset. These gains persist across different retriever architectures, including Contriever-MSMARCO, Qwen3-Embedding-0.6B, and INF-Retriever-v1-1.5B.

• Fine-Tuned RVR: Across all backbones, the FT(Di)+FT(DT) configuration consistently outperforms single-shot baselines and agentic systems, with an MRecall@100 of up to 33.7 for INF-Retriever.
• Verifier Impact: A strong LLM verifier (Qwen3-30B) approaches the performance of an oracle verifier, yielding near-optimal retrieval coverage. With an oracle, there remains headroom, indicating verification is now the principal bottleneck.

Efficiency:

Iterative agentic baselines (e.g., SearchR1, Tongyi) incur large inference costs due to repeated LLM query generation, yielding slower wall-time per query. RVR, while involving multiple retrieval and verification steps, maintains a substantially lower cost/latency profile and a smaller memory overhead compared to agentic systems.

Out-of-Domain Generalization:

RVR generalizes robustly to other multi-answer QA benchmarks, such as QUEST (requiring semantic set operation reasoning) and WebQuestionsSP. Even when only the subsequent retriever is fine-tuned on the initial QA domain, the system maintains strong performance, surpassing single-round and agentic retrieval methods in out-of-domain settings.

Component Analyses

• Verifier Budget: Performance degrades gracefully as the number of documents subject to verification shrinks, with the largest gains realized at high-verifier budgets, but significant improvements seen even at moderate settings.
• Number of Turns and Context Size: The addition of further retrieval rounds (with an effective verifier) brings diminishing returns after two iterations when using LLM-based verification, but continual improvements are observed with an oracle verifier. Optimal context sizes for appended verified documents plateau beyond six documents.
• Turn-wise Contributions: The majority of unique gold answers are recovered during the initial retrieval pass, but the second pass consistently delivers additional gold documents and covers further unique answers, with diminishing but still non-trivial gains for long-tail evidence.

Implications and Future Directions

RVR establishes the importance of iterative, verifier-in-loop retrieval for comprehensive answer coverage. Explicitly conditioning retrievers on the context of previously validated evidence, and optimizing for complementarity, leads directly to improved coverage and reduced redundancy. The modular nature of the approach enables straightforward integration with new retriever backbones, LLMs, and corpus settings.

Theory:

This formulation narrows the gap between classical IR-centric pipelines (where retrieval and verification are decoupled) and emerging agentic or LLM-centric architectures by elevating the role of verifier signal in the retrieval loop itself. It points towards a paradigm where contextual, incremental evidence gathering is first-class and optimized end-to-end.

Practice:

RVR provides improved answer completeness for open-domain QA, entity-centric search, and knowledge base population. Applications include academic/enterprise search systems and systems requiring high-recall coverage (fact-checking, knowledge discovery). Risks mirror those of any retrieval-centric pipeline: reliance on corpus integrity and potential to amplify present bias or surface misinformation in large candidate sets.

Future Work:

The framework is currently bounded by the quality/completeness of LLM-based verifiers; future developments should explore scalable, more accurate verification protocols and robust cross-domain retriever-adaptation techniques. Additional synergies may arise by coupling generative or LLM-based answer synthesis with RVR-assembled evidence sets.

Conclusion

RVR represents a step-change in multi-answer retrieval methodology. By integrating verification as an iterative, context-conditioned primitive, and optimizing retriever training for incremental evidence discovery, it offers a strong and extensible foundation for high-coverage, low-redundancy question answering in complex information environments. The release of models and code further supports adoption and extension in the broader research community.

Reference:

For full methodological and empirical detail, see "RVR: Retrieve-Verify-Retrieve for Comprehensive Question Answering" (2602.18425).