Bringing Agentic Search to Earth Observation Data Discovery

Abstract: NASA and its data centers hold thousands of geoscience datasets and tools like Worldview, Giovanni, the Science Discovery Engine, and Harmony. Finding the right one is hard even for domain experts. We present an agentic search system, deployed as a public service for the geoscience community, that takes a natural-language research query and returns the matching datasets and tools. We demonstrate that, in the era of LLMs, the latent value of knowledge graphs (KGs) can be substantially amplified through agentic search. From the NASA Earth Observation Knowledge Graph (NASA EO-KG) we derive NASA-EO-Bench, an open benchmark of 47k query-dataset pairs (21k task-based queries). A neural scorer fine-tuned on NASA-EO-Bench beats cosine and BM25 baselines. Further combining it with BM25 via score fusion raises both Recall@10 (R@10) and MRR by over 5x. On top of this supervised pipeline, we add a zero-shot agentic reranking stage that, without any additional training, lifts MRR by 28% on a stratified N=200 subset, showing that LLM reasoning is complementary to supervised retrieval.

• The paper presents a novel three-stage pipeline combining NASA tool routing, hybrid lexical-neural retrieval, and agentic LLM reranking to enhance earth observation dataset search.
• The paper leverages the NASA-EO-Bench, created from over 10,000 publications, to validate its approach, achieving a Recall@10 of 42.8% and improved MRR.
• The paper demonstrates that agentic reranking with LLM external tool calls significantly refines dataset ranking quality while acknowledging increased computational costs.

Agentic Search for Earth Observation Data Discovery: A Technical Analysis

Motivation and Context

Earth observation (EO) data discovery faces the paradox of abundance and inaccessibility. NASA and its Distributed Active Archive Centers (DAACs) maintain thousands of geoscience datasets with diverse metadata standards and access pathways, complicating efficient data retrieval. The proliferation of LLMs introduces opportunities for more natural data discovery interfaces, yet these models’ general-domain pretraining introduces systematic domain misalignment issues for tasks like geoscience dataset search. The paper "Bringing Agentic Search to Earth Observation Data Discovery" (2607.02387) addresses these core issues by developing a domain-adaptive, agentic dataset retrieval system, with strong empirical evaluations grounded in large-scale, citation-linked benchmarks.

System Architecture: Three-Stage Agentic Search Pipeline

The proposed system is organized as a three-stage agentic search pipeline (Figure 1):

Figure 1: Overview of the three-stage agentic search pipeline integrating NASA tools routing, hybrid lexical-neural retrieval, and agentic LLM reranking with external tool calls.

• Stage 1—Router on NASA Official Tools: For queries solvable via NASA's operational tools (Harmony, SDE, Worldview, Giovanni), a router dispatches and terminates the pipeline early, ensuring provenance and efficiency.
• Stage 2—Hybrid Retrieval: If official tools are insufficient, the query is mapped to NASA CMR corpus datasets by a hybrid retriever fusing BM25 lexical signals with neural semantic scorers.
• Stage 3—Agentic Reranking: The top retrieval candidates are reranked by an LLM endowed with autonomous access to external tools (web and arXiv), leveraging augmented evidence to refine ranking within the bounded candidate window.

The agentic architecture is explicitly modular, supporting independent improvements and rigorous ablation of each subsystem.

Benchmark: NASA-EO-Bench Construction and Scale

A major contribution is NASA-EO-Bench, which operationalizes citation-grounded dataset discovery evaluation at unprecedented scale. Building atop the NASA Earth Observation Knowledge Graph (EO-KG), the benchmark leverages over 10,000 highly-cited publications, generating queries by prompting LLMs on publication abstracts and using cited datasets as ground-truth pairs. This curation yields 47,654 query-dataset pairs (over 21,000 queries), stratified at publication level to prevent leakage, and providing a robust foundation for both supervised model development and comparative evaluation. This is the first benchmark in the field with both this scale and supervised training support.

Retrieval Suite: Neural-Score Correction and Lexical-Semantic Fusion

Domain-Specific Adaptation: General-purpose encoders underperform in geoscience dataset retrieval due to domain lexical mismatch and embedding-space miscalibration. The retrieval suite addresses this via:

• BM25 Lexical Anchoring: Captures high-precision matches based on instrument and variable-specific tokens, overcoming LLM embedding failures for rare scientific terms.
• Task-Adaptive Semantic Scoring (NN-SSC): Trains a pairwise MLP scorer on cached encoder embeddings, using citation-grounded positives and hard negatives to model dataset suitability. Unlike global mean-bias removal, NN-SSC captures pair-specific, nonlinear correction.
• Hybrid Fusion: Convexly combines normalized BM25 and semantic scores, with mixing coefficients set by analytic relative performance on ground-truth metrics, without validation set tuning.

Compared to fine-tuned encoders, NN-SSC adds negligible trainable parameters and provides similar or superior gains in both recall and precision.

LLM Reranking and Agentic Augmentation

LLM Reranking: After retrieval, an LLM reranks the top candidate window, using template-based prompts to judge dataset relevance. This zero-shot listwise reranking stage consistently lifts both MAP and MRR compared to retrieval-only baselines, but is fundamentally limited by the prompt context window and textual ambiguity of geoscience metadata.

Agentic Reranking: The system extends reranking by enabling the LLM to autonomously invoke web and arXiv search tools prior to ranking (agentic harness). This can resolve ambiguities not present in dataset metadata, ground decision-making in recent or emergent scientific evidence, and provide event-specific context. Empirically, agentic reranking yields a consistent MAP/MRR gain over single-shot LLM rerank baselines across multiple LLMs.

Experimental Evaluation

Strong empirical results support the architecture:

• Retrieval Suite Gains: NN-SSC + BM25 achieves Recall@10 of 42.8% and MRR of 0.292 on NASA-EO-Bench test set, more than $5\times$ above unadapted cosine baselines.
• LLM Reranking: Zero-shot LLM reranking boosts MAP and MRR on retrieval outputs for all tested LLM vendors. The gain is inversely related to the ceiling of the retrieval backbone; stronger retrieval leaves less room for rerank correction, but with higher absolute performance post-reranking.
• Agentic Reranking: The agentic harness further increases MAP/MRR over LLM rerank, with more significant gains in models that invoke external tools selectively rather than indiscriminately (e.g., Opus 4.7 vs. DeepSeek v4 pro). However, computational costs are 5-10× higher per query.

Metric Considerations

Evaluation is multi-faceted, with domain-grounded citation recall (Recall@$K$, MAP, MRR) complemented by "LLM-as-a-Judge" semantic precision, addressing the incompleteness of citation ground-truth (datasets can be relevant but uncited). Both metrics display directional consistency, but also highlight the tradeoffs in optimizing for citation co-occurrence versus semantic utility as judged by LLMs.

Limitations

• Citation Bias: The ground-truth labels reward canonical datasets and penalize non-mainstream but potentially relevant alternatives. This likely inflates evaluation on popular datasets.
• Potential Label Leakage: Given query construction, the agentic system could, in principle, rediscover the citation list via external search, which may artificially favor agentic rerank evaluation if not carefully controlled.
• Evaluation Scope: Both queries and the LLM judge are AI-generated, and human-in-the-loop validation is not present in this iteration.
• Resource-Intensity: Agentic reranking is substantially more resource-intensive, raising cost-stage tradeoffs for real-world deployments.

Implications and Future Directions

This work demonstrates that agentic search architectures can substantially improve geoscience dataset discovery, with modular neural-lexical fusion and LLM-based (including agentic) reranking providing quantifiable incremental contributions. From a practical standpoint, such systems lower the entry barrier for non-experts navigating sprawling EO data ecosystems and increase the reliability of research data selection. Theoretically, the pipeline informs how LLMs’ externally-agentic capabilities can complement knowledge graph priors and retrieval systems in highly technical, low-resource domains.

Future research will benefit from:

• Cost-aware, selective agentic tool routing strategies;
• Expansion to full-pipeline agentic operation (beyond reranking);
• Human-annotated query and judge validation for further label quality assurance;
• Extensions to broader scientific and cross-domain dataset retrieval settings using similar citation-grounded and agentic benchmarks.

Conclusion

"Bringing Agentic Search to Earth Observation Data Discovery" (2607.02387) defines a verifiable, citation-grounded framework for integrating hybrid retrieval, neural correction, and agentic LLM reranking into domain-robust EO data discovery pipelines. The system achieves substantial improvements over neural or lexical baselines and demonstrates that external tool-augmented, agentic LLMs contribute positively and measurably to dataset ranking quality in technical scientific settings. This methodology is extensible, rigorously evaluated, and provides a blueprint for further advances in agentic scientific data systems.