Beyond Parallel Sampling: Diverse Query Initialization for Agentic Search

Published 15 Jun 2026 in cs.AI and cs.IR | (2606.17209v1)

Abstract: Test-time scaling for agentic search typically increases depth (i.e., more turns and tokens per trajectory) or breadth (i.e., more parallel rollouts). Here we focus on breadth scaling, showing that standard parallel sampling yields diminishing returns, tracing this to query redundancy at the first turn. When models issue similar first queries across rollouts, the threads retrieve overlapping evidence, and subsequent turns are conditioned on this shared retrieval. We address this limitation with DivInit, a training-free intervention at the first turn. Rather than sampling k independent first queries, DivInit draws n candidates from a single call, picks k < n diverse seeds, and runs them as parallel trajectories. Across five open-weight models and eight benchmarks, DivInit consistently improves over standard parallel sampling, with average gains of five to seven points on multi-hop QA at matched compute. Code available at https://github.com/cxcscmu/diverse-query-initialization

Abstract PDF Upgrade to Chat

Authors (6)

Summary

The paper demonstrates that DivInit, a training-free strategy, significantly mitigates anchor collapse in parallel agentic search by diversifying initial queries.
It employs oversampling and Maximal Marginal Relevance to select diverse query seeds, yielding improvements up to 7 points in pass@4 scores across models.
Empirical results across multi-hop QA and open-web reasoning tasks highlight DivInit's scalability and efficiency in practical search applications.

Diverse Query Initialization for Agentic Search: An Analytical Overview of DivInit

Introduction

Agentic search in LLM systems has seen substantial gains from test-time scaling, both through increasing the depth of reasoning trajectories and expanding the number of parallel rollouts. Existing breadth scaling strategies typically employ independent parallel sampling, but this work identifies a significant inefficiency inherent to this approach: anchor collapse. In this phenomenon, parallel rollouts are conditioned on near-identical turn-1 queries, which induces correlated evidence retrieval and limits the diversity of search space exploration, thereby diminishing returns as computational budget increases.

This paper introduces DivInit ("Diverse Query Initialization") as a training-free, agent-agnostic intervention targeted at the first decision in parallel agentic search. By oversampling multiple candidate queries in a shared generation and maximizing diversity among the seeds selected to initialize agent threads, DivInit systematically mitigates anchor collapse. The paper empirically establishes the robust superiority of DivInit across five open-weight LLMs and eight benchmarks, with distinct improvements on multi-hop QA and open-web reasoning tasks.

Figure 1: A conceptual illustration of retrieval-path exploration under sequential execution, standard parallel sampling, and DivInit; DivInit leads to more divergent search-space coverage.

Anchor Collapse in Standard Parallel Agentic Search

The core diagnostic is that standard parallel sampling—drawing $k$ trajectories per input question—exhibits low turn-1 query diversity. Two metrics quantify this collapse: Query Pairwise Distance (QPD), using the mean Jaccard distance between turn-1 queries, and document overlap, the Jaccard similarity between initial retrieved document sets. Empirical traces on search benchmarks (notably GAIA) reveal failed questions cluster at low QPD and high overlap, confirming limited coverage of the evidence space.

Figure 2: QPD and inter-thread document overlap under standard parallel sampling: failed questions show low QPD/high overlap, signifying anchor collapse.

The DivInit Method: Diverse Query Pooling

DivInit amends only the first turn of agentic search, while leaving the downstream search-and-reasoning loop unchanged. The procedure involves:

Generating an oversampled pool of $n > k$ first-turn query candidates using a single LLM call (typically $n=16$ , $k=4$ ), at a fixed temperature.
Selecting a maximally diverse subset of $k$ queries using Maximal Marginal Relevance (MMR), based on token-level Jaccard distance.
Initializing each rollout with a selected seed, so that each thread explores a distinct initial retrieval pathway.

This design eliminates $k-1$ generation calls for turn-1, reducing compute for the initial step (especially as model size increases).

Main Experimental Results

Benchmarks cover both closed-domain multi-hop QA (using Wiki18 BM25 retrieval) and open-web reasoning tasks (using live SERPER web search). All results are measured under fixed token budget, reporting pass@ $k$ for group answer recall as well as single-answer metrics with the AggAgent aggregator.

Strong numerical results include:

Consistent improvements over standard parallel sampling, with average pass@4 gains of 5-7 points on multi-hop QA tasks at matched compute.
Benefit grows with model scale: For Qwen3, average gain increases from 2.8 at 1.7B parameters to 7.4 at 8B.
Marginal benefit saturates as pool size $n$ increases (Section A.1), with $n=8$ –16 sufficient for strong gains.

Performance curves for both multi-hop QA and open-web tasks show DivInit's advantage persists for alternative choices of $k$ and aggregation strategies.

Figure 3: Multi-hop QA pass@ $n > k$ 0 curves (k = 1 to 8); DivInit outperforms standard parallel at all scales and datasets.

Figure 4: Open-web reasoning pass@ $n > k$ 1; DivInit maintains clear advantage across model and task combinations.

Mechanistic Analysis of Diversity Persistence and Benefit

DivInit significantly increases the turn-1 QPD distribution (often from $n > k$ 2 under standard parallel sampling to $n > k$ 3). Notably, Across-Thread Distance (ATD)—diversity in queries issued at subsequent turns—remains tightly correlated to initial QPD, indicating that early diversity persists and guides agentic branches into divergent evidence regions throughout the trajectory.

Figure 5: Turn-1 QPD distribution is substantially shifted by DivInit; per-question QPD and subsequent ATD are tightly correlated, confirming persistent search-space separation.

Ablation studies show:

Extending pool selection beyond the first turn offers negligible additional gains: Early separation is sufficient.
Explicit diversity selection via MMR reliably outperforms random subsampling from the candidate pool.
Simply increasing temperature ( $n > k$ 4) under standard parallel sampling increases QPD but does not close the performance gap versus DivInit.
Figure 6: Pass@4 as temperature varies; higher $n > k$ 5 boosts standard parallel sampling but does not reach DivInit's efficacy.

Case Studies and Limitations

Trajectory-level analysis on GAIA demonstrates that standard parallel rollouts typically issue queries that are under-differentiated, resulting in the same failures across all threads. In contrast, DivInit seeds probe different subquestions or entry points, enabling at least one thread to explore a more productive path.

Certain failure modes remain: when very precise lexical matching is required (e.g., extracting an exact string from a script), increased diversity may diffuse effort over less-relevant space. For small models, benefit of diversification is limited by low underlying retrieval and reasoning capacity.

Implications and Future Directions

The central implication is that standard breadth scaling under-parallelizes over the actual solution space: most of the compute is spent on highly correlated reasoning chains, which limits oracle and aggregated answer recall. The DivInit mechanism is training-free, compatible with any agentic loop, and its overhead is typically offset by reduced generation calls for large models.

This design suggests several advances:

RL Improvements: In reinforcement learning for agentic search, initial anchor collapse may yield low-variance exploration and weak credit assignment; initializing diverse rollouts prior to group-based policy updates could address this.
Aggregation: While pass@ $n > k$ 6 ceiling improves, optimal aggregation of diverse trajectories into a single response remains a challenging problem. Efforts to exploit structure among diverse rollouts for aggregation (beyond majority or reward signals) are warranted.
Domain Extension: While this study focuses on search-agent settings, anchor collapse may extend to other domains (e.g., sequential tool-use, program synthesis), necessitating analogous interventions.

Conclusion

DivInit pinpoints and remedies the anchor collapse phenomenon in parallel agentic search by ensuring diverse initialization of trajectories. The improvements are robust across models, benchmarks, and aggregation strategies, and point towards a general strategy for maximizing inference-time compute utility in agentic LLM systems. Exploiting the structural diversity enabled by DivInit for advanced aggregation and extending the approach to RL or non-search settings represent fruitful avenues for future research.

Figures

Figure 1: A conceptual illustration of retrieval-path exploration under sequential execution, standard parallel sampling, and DivInit; DivInit leads to more divergent search-space coverage.

Figure 2: Turn-1 QPD and inter-thread document overlap under standard parallel sampling (Qwen3-8B, $n > k$ 7); failed questions cluster at low QPD/high overlap.

Figure 6: Pass@4 under standard parallel sampling at varying $n > k$ 8 (Qwen3-8B); DivInit maintains an advantage not remedied by increased sampling temperature.