Knowing You Don't Know: Learning When to Continue Search in Multi-round RAG through Self-Practicing

Abstract: Retrieval Augmented Generation (RAG) has shown strong capability in enhancing LLMs' knowledge and reducing AI generative hallucinations, driving its widespread use. However, complex tasks requiring multi-round retrieval remain challenging, and early attempts tend to be overly optimistic without a good sense of self-skepticism. Current multi-round RAG systems may continue searching even when enough information has already been retrieved, or they may provide incorrect answers without having sufficient information or knowledge. Existing solutions either require large amounts of expensive human-labeled process supervision data or lead to subpar performance. This paper aims to address these limitations by introducing a new framework, SIM-RAG, to explicitly enhance RAG systems' self-awareness and multi-round retrieval capabilities. To train SIM-RAG, we first let a RAG system self-practice multi-round retrieval, augmenting existing question-answer pairs with intermediate inner monologue reasoning steps to generate synthetic training data. For each pair, the system may explore multiple retrieval paths, which are labeled as successful if they reach the correct answer and unsuccessful otherwise. Using this data, we train a lightweight information sufficiency Critic. At inference time, the Critic evaluates whether the RAG system has retrieved sufficient information at each round, guiding retrieval decisions and improving system-level self-awareness through in-context reinforcement learning. Experiments across multiple prominent RAG benchmarks show that SIM-RAG is an effective multi-round RAG solution. Furthermore, this framework is system-efficient, adding a lightweight component to RAG without requiring modifications to existing LLMs or search engines, and data-efficient, eliminating the need for costly human-annotated mid-step retrieval process supervision data.

• The paper introduces SIM-RAG, a framework that enhances multi-round RAG systems' self-awareness for complex reasoning by generating synthetic data through 'self-practicing'.
• SIM-RAG trains a lightweight Critic model using this synthetic data to assess information sufficiency at each step and guide retrieval decisions, preventing over-confidence.
• Empirical validation shows SIM-RAG outperforms baseline methods on multi-hop reasoning tasks, demonstrating that self-awareness improves performance without needing larger language models.

An Analytical Overview of SIM-RAG: Enhancing Multi-round Retrieval Augmented Generation

LLMs have exhibited substantial potential in various reasoning tasks, yet when tasked with complex, multi-round retrieval scenarios, traditional methods often fall short of human-level performance. The paper, "Knowing You Don't Know: Learning When to Continue Search in Multi-round RAG through Self-Practicing," introduces SIM-RAG, a framework designed to strengthen retrieval augmented generation (RAG) systems, specifically enhancing self-awareness for complex reasoning tasks that necessitate multiple rounds of information retrieval.

Main Contributions

SIM-RAG addresses the tendency of current multi-round RAG systems to either over-retrieve or provide confident answers based on insufficient information. This framework employs process supervision, inspired by human meta-cognition, integrated through a novel approach dubbed Self-Practicing. This method generates synthetic data that reflects a model's inner reasoning trajectory—its "inner monologue"—which allows for learning the nuanced domain-specific reasoning paths without costly human-annotated data.

Self-Practicing and Critic Model

The self-practicing stage involves enabling the RAG system to simulate human-like reasoning by continual assessment and retrieval. During this process, the model generates answers and rationalizes its decisions, which are labeled either as accepted or rejected based on their success in reaching correct outcomes. This synthetic data forms the basis for training a lightweight Critic model—separate from the LLM itself—to evaluate the sufficiency of information retrieved at each round and guide retrieval decisions effectively.

The Critic, a task-specific yet lightweight discriminative model, acts as an external supervisor, assessing the Reasoner's predictions. It is a pivotal element, trained to interpret reasoning paths and coherence without the necessity of knowledge embedded within the LLM. The trained Critic offers high accuracy in rejecting incorrect answers, especially on tasks requiring multi-hop reasoning, thereby preventing over-confidence and minimizing the risk of hallucination.

Empirical Validation and Analysis

Evaluation of SIM-RAG is conducted across traditional RAG datasets: TriviaQA for single-hop reasoning, and HotpotQA and 2WikiMultiHopQA for multi-hop tasks. The results demonstrate that SIM-RAG consistently surpasses established RAG and prompting-based systems. Notably, with EM scores reaching up to 77.5% on TriviaQA, SIM-RAG highlights a significant departure from over-confident responses produced by standard methods.

Moreover, comparing Critic model sizes and performances reveals that even the lightweight version markedly enhances reasoning outcomes, supporting the idea that reflective reasoning does not necessitate larger model footprints, thereby balancing performance and computational efficiency.

Implications and Future Directions

SIM-RAG's enhancements in multi-round RAG show promising implications for domains reliant on accurate, iterative reasoning, and highlight future paths for AI advancements. It sets a precedent for separating reasoning and critique processes—allowing each component to specialize—maximizing LLMs' strengths without intruding on their internal architecture.

Potential future work includes expanding the Critic's feedback mechanisms to support more diverse reasoning tasks, investigating domain adaptations leveraging the synthetic data generation approach, and exploring more dynamic retrieval methods that can optimize multi-hop reasoning scenarios further.

In summary, SIM-RAG provides a pragmatic framework for strengthening the self-awareness of RAG systems, suggesting an evolution away from monolithic LLMs towards modular, adaptive architectures. This underscores the continuing journey in AI research towards systems that not only 'think' but also possess the discernment of knowing their own knowledge limitations, thus furthering the quest for true intelligence.