EmbeddingRWKV: Efficient State Retrieval

• EmbeddingRWKV is a state-centric retrieval framework that caches intermediate neural states for efficient document reranking.
• It reduces computation by processing only query tokens post state-caching, yielding significant speedups compared to full model evaluations.
• Empirical results indicate that using uniform layer sampling retains over 98% performance while dramatically lowering memory and compute demands.

State-centric retrieval refers to information access paradigms in which retrieval and reasoning operate over structured “states” of either the world, documents, or memory, rather than only unstructured text or flat embeddings. A “state” may denote the causal outcome of events, the latent state-space of a neural model, or a structured history over time and space. Unlike conventional IR that aggregates relevance from keyword overlap or static vector similarity, state-centric retrieval frameworks aim to reconstruct, manipulate, or directly utilize such states to answer queries, often with deep integration into the downstream reasoning or decision process.

1. Formal Definitions and Core Paradigms

State-centric retrieval admits multiple formalizations, depending on the application context:

• Action-Centered/World-State IR: Here, the world’s state evolves through actions/events described in narratives; the retrieval process reconstructs a possible state (“fluents”) at query time and selects documents based on whether their implied end-state entails the query (Balduccini et al., 2019).
• State-Space Model (SSM) or “Neural State” Retrieval: The “state” is the latent memory, e.g., the hidden vectors from stateful neural architectures (e.g., RWKV, SSM). Queries are answered by minimizing model uncertainty as a function over a mixture of precomputed document states, directly via in-context optimization (Becker et al., 13 Jun 2025, Hou et al., 10 Jan 2026).
• Spatiotemporal/Embodied State Retrieval: States comprise temporally and spatially indexed object or scene records, enabling agents (e.g., robots) to condition queries or decisions on past object placements, attributes, or contexts (Chen et al., 18 Nov 2025).

Across these settings, a common thread is that retrieval is fundamentally about selecting, weighting, or traversing states so as to most directly resolve the user’s informational or decision objective, rather than relying solely on surface similarity to the query text.

2. Formal Methodologies and Retrieval Objectives

• Event-State Query Formalization (Balduccini et al., 2019): Documents provide sequences of actions; the task is to find all documents where, after simulating these actions including ramification and non-determinism, a certain query fluent holds. This is formalized in the action language $\mathcal{A}\mathcal{L}_{IR}$ and operationalized using Answer Set Programming (ASP), encoding actions, fluents, effects, and inertia. Retrieval is defined as an (F, s) search for support of the query with minimum semantic cost.
• State Mixture Minimization in LLMs/SSMs (Becker et al., 13 Jun 2025): Given a document store $\mathcal{D} = \{d_i\}$, each with a precomputed state $h_{d_i}$ and a query $q$ with state $h_q$, define a weight vector $\alpha \in [0,1]^N$ to form a weighted state $\bar h(\alpha) = \sum_i \alpha_i h_{d_i}$. Retrieval seeks $\hat\alpha = \arg\min \mathcal{L}(\alpha)$, where the loss $\mathcal{L}(\alpha)$ is the negative log-likelihood of generating the query given the combined state. This is solved efficiently by gradient-based updates (as in RICO), where gradient inner products with document states provide retrievability scores; top-$k$ selection approximates the optimization.
• Spatiotemporal Retrieval Formalism (Chen et al., 18 Nov 2025): Each observation is a tuple $m_i = (t_i, x_i, c_i, o_i)$, representing time, spatial pose, attribute embedding, and raw data. A retrieval query parses the language instruction $\ell$ into subgoals and returns indices that jointly satisfy attribute, spatial, and temporal requirements. The search (as in STAR) operates by issuing memory queries and spatial actions in a unified loop.

3. Architectures and Computational Properties

Context	State Representation	Retrieval Mechanism	Model class
Event-world/ASP	Extended fluents, ASP	Program synthesis + answer set	Logic/ASP
SSM/LLM (RICO, RWKV)	Latent model states	State-space optimization	State-space model
Spatiotemporal/Robotics	$(x, t, \alpha)$ tuples	Attribute + spatial + temporal NN	Memory + LLM agent

Key architectural traits:

• In (Becker et al., 13 Jun 2025), SSMs or linear-attention transformers precompute document states; RICO then performs iterative gradient-based retrieval using only these cached states and the incoming query.
• EmbeddingRWKV (RWKV-based state-centric retrieval) caches intermediate layer states for each document; reranking only passes query tokens through the model, massively reducing computational overhead relative to transformer cross-encoders (Hou et al., 10 Jan 2026).
• For event-driven/logical state IR, the entire document narrative is compiled to a logic program whose answer sets are exhaustively checked for entailment with respect to query conditions (Balduccini et al., 2019).

4. Efficiency, Compression, and Scalability Properties

• Reusable States and Layer Compression: EmbeddingRWKV shows that storing only a uniformly sampled subset ($\sim25\%$) of layers’ state matrices maintains $>98\%$ full-model performance (Hou et al., 10 Jan 2026). This reduces the storage (and memory read) required for state-centric reranking, facilitating larger-scale deployments.
• Decoupling Reranking Cost from Document Length: By initializing the model state with the precomputed document state, reranking cost is made a function of query length rather than document length (since only the query tokens are processed), yielding speedups of $5.4\times$–$44.8\times$ as document length grows (Hou et al., 10 Jan 2026).
• Gradual vs. Discrete Retrieval: RICO shows that a single gradient step achieves retrieval quality within $5\%$ of full multi-step optimization, and that the method is robust to warm starts from standard sparse retrievers (Becker et al., 13 Jun 2025).
• Memory Bottlenecks and Mitigation: Sequential state caching poses inherent scalability limitations; subsampling, random projections, and selective layer retention are effective mitigation strategies (Becker et al., 13 Jun 2025, Hou et al., 10 Jan 2026).

5. Applications and Evaluation Results

• Short- and Long-Context QA: On MS MARCO, HotpotQA, MuSiQue, 2WikiMultihopQA, and TriviaQA, RICO achieves nDCG@10 $\approx 0.64$ (Mamba2-130m), outperforming standard BM25 and approaching large dense retrieval models, with superior generation F1 on chain-of-thought and long-context regimes (Becker et al., 13 Jun 2025).
• General IR and Multilingual Tasks: On MTEB and NanoBEIR, EmbeddingRWKV matches or exceeds standard retrieval and reranking baselines at lower computational and memory cost (Hou et al., 10 Jan 2026).
• Robotics/Embodied Retrieval: In the STAR framework, integrating spatial, temporal, and attribute indices into state-centric retrieval drastically improves physical object search, achieving realistic mode success rates of $0.77$ (visible search), $0.61$ (attribute, spatio-temporal), and $0.78$ (real-world deployment, attribute) (Chen et al., 18 Nov 2025).
• Action-Centered IR: Logic-driven state-centric retrieval is shown to be computationally feasible for moderately sized narrative domains, with average query resolution times on the order of $1$–$10$ seconds for practical stories (Balduccini et al., 2019).

6. Theoretical Extensions, Open Problems, and Practical Implications

• Theory: State-centric retrieval unifies and generalizes classical IR, memory-augmented generation, and causal reasoning, with tight connections between state-space optimization and classical leave-one-out and perplexity-based retrieval objectives (Becker et al., 13 Jun 2025).
• Practice: State-centric paradigms allow dynamic compute–quality tradeoffs (e.g., number of gradient steps or amount of state cached), are model-agnostic at inference, and lower deployment complexity in RAG systems by allowing a single backbone for retrieval and reranking (Hou et al., 10 Jan 2026).
• Limitations: These methods depend on high-quality initial state extraction and, for logic-based paradigms, on robust NLP-to-logic translation. Scalability becomes critical in very long document histories or persistent open-world robotic deployments, requiring memory management or state compression (Balduccini et al., 2019, Chen et al., 18 Nov 2025).
• Open questions: Adaptive, data-dependent state selection, further theoretical analysis of state overlap and redundancy, and reinforcement-learning driven refinement of retrieval policies represent ongoing research avenues (Chen et al., 18 Nov 2025, Hou et al., 10 Jan 2026).

State-centric retrieval—across event-driven, neural, and embodied settings—constitutes a foundational shift from flat match scoring to deep, structured utilization of system states, coupling retrieval directly to reasoning and action within diverse AI and IR systems.