Multi-Step Agentic Retrieval

• Multi-step agentic retrieval is a dynamic approach that decomposes complex queries into iterative sub-queries using autonomous LLM-based agents.
• It integrates adaptive decision-making and multi-tool orchestration to refine evidence gathering and address context-dependent information needs.
• Researchers leverage multi-hop reasoning and structured graph traversal to enhance accuracy and robustness in knowledge-intensive tasks.

Multi-step agentic retrieval refers to the use of autonomous, often LLM-based agents to orchestrate iterative, adaptive information-seeking processes—interleaving targeted sub-query generation, tool invocations, and reasoning—toward the dynamic construction of a complete answer state. Unlike traditional single-pass retrieval-augmented generation (RAG), multi-step agentic approaches treat the search for information as a sequential decision process, leveraging intermediate reasoning, multi-hop knowledge traversal, and the full agency of the underlying agent. Such systems have advanced core performance in domains where information needs are complex, context-dependent, and require structured or multi-hop reasoning (Lelong et al., 22 Jul 2025, Zhang et al., 2024, Li et al., 13 Jul 2025).

1. Foundations and Motivation

Traditional RAG pipelines perform a single fixed retrieval: a static query $q$ yields a top-$k$ set of documents from a corpus, which are then used as passive context for answer generation. This approach is brittle for knowledge-intensive tasks—such as multi-hop QA, scientific data exploration, or regulatory compliance—where evidence must be found across heterogeneous sources and discovered incrementally, often conditioned on partial hypotheses or earlier reasoning steps (Lelong et al., 22 Jul 2025, Zhang et al., 2024, Li et al., 13 Jul 2025).

Multi-step agentic retrieval remedies these limitations by empowering agents to:

• Decompose complex queries into sub-questions adaptively, issuing new queries conditioned on prior results.
• Select and invoke specialized tools (textual, tabular, knowledge graph, database, etc.) as actions within a planning loop.
• Fuse retrieved evidence with ongoing reasoning, continually updating their internal knowledge state.
• Refine, verify, and synthesize answers iteratively, only terminating when informational sufficiency is reached.

The result is a dynamic, policy-driven interplay between search and reasoning, allowing high accuracy, coverage, and robustness in domains otherwise resistant to static retrieval (Lelong et al., 22 Jul 2025, Zhang et al., 2024, Li et al., 13 Jul 2025).

2. Core System Architectures and Formal Models

Most agentic retrieval engines implement the following layered structure (see (Lelong et al., 22 Jul 2025) for INRAExplorer):

A. System Layers

1. User & Prompting: The user poses a question in natural language; the LLM-based agent plans actions and maintains chain-of-thought.
2. Agent Orchestrator: Maintains agent state (query history, retrieved passages $R_{0..t}$, actions $A_{0..t}$, and context embedding $c_t$), exposes tools, and routes action calls.
3. Retrieval Tools: Includes hybrid search (dense + sparse), structured graph traversal (via knowledge graphs), and macro-tools for expert/entity/keyword identification.
4. Knowledge Base: Consists of both vector stores (e.g., Qdrant, Weaviate) for text chunks and rich knowledge graphs (e.g., Neo4j) encoding entities and relationships.

B. Agentic Decision Process

Formally, for discrete time steps $t = 0, 1, ..., T$:

• Agent state $s_t$ encodes query/action/retrieval history and latent context embedding $c_t$.
• At each $t$, the agent chooses an action $a_t$ (tool invocation including arguments) from the available set $\mathcal{A}_t$ using a softmax policy over LLM logits:

$$\pi(a_t \mid s_t) = \frac{\exp(\ell_t(a_t))}{\sum_{a'\in \mathcal{A}_t} \exp(\ell_t(a'))}$$

• The orchestrator executes $a_t$, receives retrieval $r_t$, updates context via fusion (e.g., cross-attention):

$$c_{t+1} = \text{TransformerBlock}(c_t, [\text{Enc}_\text{text}(r_t); \text{Enc}_\text{graph}(r_t)])$$

• State is incrementally aggregated: $s_{t+1} = \{ s_t \cup \{a_t, r_t\},\ c_{t+1} \}$.
• Loop continues until the agent emits "Finish" or hits $T_\text{max}$.

C. Multi-Hop Reasoning Over Graphs

For knowledge graph $G = (V, E, \Phi)$:

• Nodes $v\in V$ (authors, publications, concepts, etc.) carry type and metadata.
• Edges $e=(u \to v)$ are typed (e.g., AUTHORED_BY, DESCRIBES) with weights $w_r > 0$.
• Paths $p = (u_0 \to u_1 \to ... \to u_k)$ are scored:

$$\text{Score}(p) = \prod_{i=1..k} w_{r_i} \cdot \text{sim}(\text{Emb}_\text{path}(p),\ \text{Emb}_\text{query})$$

where $\text{sim}$ is cosine similarity.

• Retrievals can specify max-hop depth $H$; returned subgraphs are weighted and filtered by path score.

3. Agentic Control and Multi-Tool Orchestration

Autonomous query decomposition and tool selection are hallmark features:

• The LLM agent explicitly plans sequences of retrieval actions—e.g., alternating between document search, entity exploration, and concept lookup—based on context and intermediate answers.
• Macro-tools encapsulate multi-step tool combinations for specialized reasoning (e.g., "IdentifyExperts" internally chains publication and graph search plus scoring).
• Each retrieval augments context; information is fused and reflected upon before further actions are chosen.
• Tool usage and action selection are driven by learned or prompt-engineered policies, often using chain-of-thought or structured reasoning instruction.

Pseudocode sketch, as formalized in (Lelong et al., 22 Jul 2025):

q0 = user_question s0 = initialize_state(query=q0) t = 0 while True: logits = LLM(prompt=render_prompt(s_t)) a_t = select_action(logits, available_tools(s_t)) if a_t.name == "SearchPublications": r_t = call_SearchPublications(a_t.query, top_k=5) elif a_t.name == "SearchGraph": r_t = call_SearchGraph(a_t.cypher) # ... other tools ... s_{t+1} = update_state(s_t, a_t, r_t) if a_t.name == "Finish" or t >= T_max: break t += 1 final_answer = LLM(prompt=render_synthesis_prompt(s_{t+1})) return final_answer

This paradigm generalizes across agentic IR frameworks, fusing per-step state, memory buffers, action selection, and recovery from uncertainty or failure (Zhang et al., 2024).

4. Multi-Step Evaluation: Metrics and Benchmarks

Evaluating multi-step agentic retrieval demands metrics that reflect not only single-answer correctness but also the fidelity and sufficiency of the reasoning process:

Metric	Description	Citation
Hop Accuracy	Fraction of intended relation "hops" retrieved in KG traversal	(Lelong et al., 22 Jul 2025)
Answer Completeness	Proportion of required elements (authors, projects) present in synthesized answer	(Lelong et al., 22 Jul 2025)
Precision/Recall	For subgraph or passage extraction; measures both relevance and coverage	(Lelong et al., 22 Jul 2025)
End-to-end Latency	Average wall-clock time per query (with std)	(Lelong et al., 22 Jul 2025)
Agent Cost	Number of LLM tokens consumed, number of tool calls	(Lelong et al., 22 Jul 2025)

Development of domain-specific evaluation benchmarks—with expert co-design—remains essential, as standard RAG metrics under-represent the complexity encountered in multi-hop, agent-driven queries.

5. Comparative Perspectives: Agentic vs. Traditional Retrieval

Conventional RAG systems operate as single-pass, fixed pipelines—querying once, presenting retrieved items, and requiring extensive manual engineering for new domains. Multi-step agentic retrieval diverges profoundly (Zhang et al., 2024, Li et al., 13 Jul 2025):

• Adaptivity: Agents orchestrate multi-turn interactions, steering their own tool usage and context updates to reach non-trivial information states.
• Reasoning Depth: Agents can carry out complex, step-wise hypotheses—decomposing queries, issuing iterative retrievals, and re-planning based on intermediate evidence.
• Expressivity: Multi-tool and multi-hop capabilities permit workflows over graphs, text, and structured data, enabling sophisticated multi-entity, multi-relation answers.

Agentic IR expands the classical IR goal—from satisfying static queries to actively achieving user-specified information states through dynamic, context-aware decisions (Zhang et al., 2024).

6. Practical Implications and Research Direction

Integrating multi-step agentic retrieval changes both engineering and research challenges:

• System Complexity: Multi-level agent orchestration and tool management require advanced state tracking and resource control.
• Domain Adaptation: Agents must be customized to the structure and relationships of domain corpora or knowledge graphs, often necessitating bespoke tool suites and evaluation protocols.
• Evaluation: Standard metrics may be insufficient; agentic retrieval necessitates new metrics that can account for path accuracy, completeness, effort/cost, and reasoning quality (Lelong et al., 22 Jul 2025).
• Open Challenges: Efficient planning (to minimize cost/latency), multimodal tool integration, trustworthy retrieval in adversarial settings, and compositional benchmarking are active areas of research (Li et al., 13 Jul 2025).

The INRAExplorer system for scientific data (Lelong et al., 22 Jul 2025), and agentic IR as conceptualized in (Zhang et al., 2024, Li et al., 13 Jul 2025), exemplify the field's movement toward autonomous, self-reflective, and multi-tool agents capable of robust, explainable, and deep information retrieval in real-world, knowledge-intensive settings.