MemAgent: Memory-Augmented LLM Architectures

• MemAgent is a class of memory-augmented agent architectures that enable LLMs and mobile agents to process long contexts and sustain personalized dialogues.
• It employs reinforcement learning and fixed-window memory updates to achieve linear O(N) computational scaling with high accuracy.
• Implementation spans multi-layer controllers, modular memory libraries, and hardware-adapted engines to enhance efficiency, retrieval, and personalization.

MemAgent denotes a class of memory-augmented agent architectures designed to enable LLMs and mobile agents to efficiently process long-context inputs, sustain personalized and context-aware dialogue over extended horizons, and operate within computational and storage constraints through principled memory management strategies. Implementations span RL-trained segmental memory overwrite schemes, hierarchical multi-layer memory controllers, hardware-adapted vector databases for mobile SoCs, and agentic meta-control pipelines leveraging reflective reasoning for retrieval and answer synthesis.

1. Streaming Memory Overwrite Architecture and RL Optimization

MemAgent architecture, as introduced in "MemAgent: Reshaping Long-Context LLM with Multi-Conv RL-based Memory Agent" (Yu et al., 3 Jul 2025), wraps a transformer-based LLM with a fixed-length memory slot and processes arbitrarily long sequences via text segmentation. The workflow proceeds as follows:

• The document of length $N$ tokens is partitioned into $K$ contiguous segments $c^1, c^2, ..., c^K$ with each segment $\leq C$ tokens.
• At segment $k$, the agent receives the problem statement $q$, previous slot $m^{k-1}$, and segment $c^k$, generating an updated memory $m^k$ via the base LLM.
• This overwrite strategy allows the agent to summarize or retain salient information per segment, replacing the memory slot rather than appending.

The inference pipeline is strictly streaming: each prompt includes only $(q, m^{k-1}, c^k)$, producing $m^k$, with the final answer generated from $(q, m^K)$. This fixed-window approach guarantees $O(N)$ compute as opposed to standard context-concatenation approaches with $O(N^2)$ cost.

Reinforcement learning is essential for effective memory utilization. The update process is cast as an MDP with states $(q, m_{k-1}, c_k)$ and actions as the memory-update outputs. Only the terminal answer step incurs reward, propagated back to the memory updates via a Multi-Conversation extension of the DAPO algorithm, which generalizes the PPO/GRPO framework. The objective aggregates advantages across all dialogues, broadcasting the episode reward uniformly.

Empirical results on synthetic long-context QA tasks demonstrate <5% loss up to 3.5M tokens of input, with accuracy $\geq$ 95% on 512k-token RULER test tasks. RL training is critical; fixed-memory agents without end-to-end RL degrade substantially past 896k tokens (Yu et al., 3 Jul 2025).

2. Multi-Layer Memory Controllers for Dialogue and Personalization

The Mixed Memory-Augmented Generation (MMAG) pattern organizes agent memory into five interoperating layers, each exposed as an independent service with retrieval, scoring, and prioritization logic unified under a central Memory Controller (Zeppieri, 1 Dec 2025):

1. Conversational Memory: Append-only logs preserving session-level dialogue, responsible for reference resolution, topic tracking, and anaphora handling.
2. Long-Term User Memory: Stores persistent, user-specific facts (preferences, biographical traits), leveraged for personalization and adaptive prompt engineering.
3. Episodic & Event-Linked Memory: Timestamped event entries and habitual pattern cues; supports reminders and proactive nudges via scheduled triggers.
4. Sensory & Context-Aware Memory: Ingests situational signals (location, device state, time) to ground interaction context.
5. Short-Term Working Memory: Ephemeral buffers for intra-task variables and intermediate reasoning chains; supports multi-step problem solving.

Each memory candidate $m_i$ from layer $\ell$ receives a retrieval score: $$\mathrm{score}(m_i) = \alpha_\ell \cdot \mathrm{sim}(q, m_i) + \beta_\ell \cdot \mathrm{decay}(t_{\text{now}} - t_i) + \gamma_\ell \cdot \mathrm{priority}(m_i)$$ with layer-specific hyperparameters $\alpha_\ell, \beta_\ell, \gamma_\ell$. Pruning and conflict resolution are solved as an integer-knapsack problem, maximizing cumulative score under a token budget constraint.

Coordination policies integrate recency, personalization, and task relevance via composite ranking functions and maintain coherence by enforcing minimal inclusion of working memory for complex operations. Capacity management applies aggressive pruning by ascending importance to stay within tight context windows (e.g., $T_{\mathrm{max}} \approx 90$k tokens).

Production deployment in the Heero agent utilizes the conversational and long-term memory layers, achieving a 20% increase in four-week user retention and 30% increase in average conversation length, with negligible latency overhead (median response $\approx$ 750ms) and traceable privacy via envelope encryption and audit logs (Zeppieri, 1 Dec 2025).

3. Unified Modular Memory Libraries and Retrieval Models

MemEngine provides a modular framework and implementation library unifying diverse memory models for LLM-based agents (Zhang et al., 4 May 2025). Its architecture consists of:

• Memory Functions: Encoders, retrievers, summarizers, reflectors, forgetters, and direct LLM interfaces, each with well-defined APIs.
• Memory Operations: Composition of functions into store, recall, manage, and optimize pipelines, supporting trajectory-level meta-learning and reflection.
• Memory Models: Pre-built variants—FUMemory (full concatenation), STMemory (sliding window), LTMemory (semantic vector index), GAMemory (LLM-judged generative agent), MBMemory (multi-layer summarization with dynamic forgetting), SCMemory (LLM-controlled minimal recall), MGMemory (hierarchical MemGPT FS-like), RFMemory (meta-learned retrieval), MTMemory (semantic tree/traversal).

Retrieval is often embedding-based, e.g., cosine similarity $\mathrm{sim}(\mathbf{q},\mathbf{m}_i)$ for long-term memory, or hybrid scoring with LLM-judged importance for generative agents.

Empirical evaluation shows MBMemory achieving the highest Precision@5 (0.83), with GAMemory (0.78) and LTMemory (0.72) trailing, and corresponding agent task success rates: MBMemory (85%), GAMemory (79%), LTMemory (74%), FUMemory (65%), STMemory (68%). Token cost and runtime overheads are also benchmarked.

Plugin mechanisms permit rapid addition of new memory functions or models, and utilities include persistent storage backends, visualization tools, and scalable remote deployment via FastAPI. Current limitations include text-only support, external LLM dependency for some management tasks, and encoder bias; future work seeks multi-modal memory and privacy-preserving stores (Zhang et al., 4 May 2025).

4. Hardware-Adapted Memory Engines for Mobile Agents

The Agentic Memory Engine (AME) addresses the distinct constraints of on-device agents on smartphone SoCs, co-designing memory access patterns and data structures with mobile hardware properties (Zhao et al., 24 Nov 2025). Key architectural elements:

• Matrix Compute Pipeline: Vector similarities mapped to accelerator-native GEMMs, exploiting multi-level memory hierarchies (SRAM, NPU-TCM, GPU scratchpads) and data layout adaptation (FP32→FP16, tile-major 32×64).
• Vector Index and Scheduler: IVF index tuned for NPU tile shape, windowed batch scheduler distributing tasks across CPU, GPU, and NPU according to scenario-specific templates (query, update, index-rebuild, hybrid).

Algorithms support efficient top-$K$ searches, insertions, and cluster-based index rebuilds with complexity controlled by $O(d \log N)$ for queries, $O(C d)$ for insertions, and $O(T M C d)$ for rebuild cycles.

Experimental results on Snapdragon 8-series show AME delivers up to 1.4× higher QPS at matched Recall@10, 7× faster index build, and 6× higher concurrent insertion throughput than HNSW baselines. Double-buffering, SMT thread scheduling, and unified shared-memory fabrics minimize bandwidth and idle resource cost. Lessons learned emphasize data layout and scheduling as critical for SoC-bound environments; extensions include adaptive recall, quantized embedding storage, and multimodal key support (Zhao et al., 24 Nov 2025).

5. Episodic Retrieval-Augmented Planning in Mobile Task Agents

MemRAG, the memory module within MobileRAG, enables LLM-based mobile agents to leverage historical solutions in complex, real-world app tasks (Loo et al., 4 Sep 2025). The memory is structured as a key–value store:

• Keys: Embeddings of prior user queries.
• Values: Tokenized action sequences corresponding to verified solution paths.

Retrieval employs cosine similarity; when a new query embeds within threshold ($\tau = 0.8$), the best-matching sequence is reused (direct replay for exact match). Otherwise, retrieved snippets are prefix-injected into the LLM prompt to bootstrap complex plans. Memory is updated on successful execution; optional soft-attention fusion aggregates multiple memories for injective contexts.

Evaluation on MobileRAG-Eval demonstrates the effect of MemRAG: Action Fidelity improves (AF: 88.7%→91.1%), Task Completion Ratio rises (TCR: 93.8%→97.6%), and Task Success Rate gains (TSR: 86.7%→93.3%). Average operational sequences drop by 2.4 steps relative to no-memory baselines. Limitations include static similarity thresholding, unbounded store growth, lack of fine-tuned fusion, and risk of homogenization precluding adaptation (Loo et al., 4 Sep 2025).

6. Closed-Loop Reflective Retrieval for Evidence-Grounded Answering

MemR³ abstracts agentic memory-retrieval as a closed-loop decision process, augmenting standard external memory stores with a router-driven pipeline comprising Retrieve, Reflect, and Answer nodes (Du et al., 23 Dec 2025). The router selects actions based on the current agent state—including accumulated evidence set $\mathcal{E}_{k-1}$, current gap $\mathcal{G}_{k-1}$, and a cap on reflect-streak to avoid indecision loops. The evidence-gap tracker transparently computes required fact coverage against the ideal requirement space $R(q)$, halting when $|\mathcal{G}_k| = 0$ or upon exceeding maximum iterations.

Integration is plug-and-play with RAG (vector-indexed chunks) or Zep (temporal knowledge graphs), requiring only query/response adaptation and tracking of snippet usage. Empirical tests on the LoCoMo benchmark yield consistent improvements in LLM-as-a-Judge scores (+7.29 pp over baseline RAG, +1.94 pp over Zep). Substantial gains are observed in multi-hop, temporal, and open-domain categories.

Reflective steps enable focused gap-filling and iterative evidence correction. Traceable $\mathcal{E}_k, \mathcal{G}_k$ trajectories provide human-readable auditability of support and remaining uncertainties; potential extensions involve neural routers and multi-modal retrieval (Du et al., 23 Dec 2025).

7. Comparative Summary and Research Outlook

MemAgent architectures consistently demonstrate advanced memory control capabilities in LLM agents and mobile environments, enabling scalable processing of long or multi-turn contexts, fast and prioritized retrieval of historical or semantic information, and the capacity for continual adaptation through RL or agentic reasoning pipelines.

Performance benchmarking confirms true linear scaling (O(N)) in segmental overwrite agents with RL-trained memory, large retention and engagement gains in deployed MMAG systems, superior query and update throughput in hardware-adapted engines, and heightened practical success rates in retrieval-augmented mobile task agents.

Open challenges include extension to multimodal memory (images, audio, sensor data), lifelong dynamic embedding learning, fine-grained user memory control interfaces, privacy and fairness mitigation, and maintaining sub-second retrieval latencies as memory taxa scale. A plausible implication is that the modularity and controller-based design of recent MemAgents presage more autonomous, explainable, and lifelong-learning conversational agents (Yu et al., 3 Jul 2025, Zeppieri, 1 Dec 2025, Zhao et al., 24 Nov 2025, Zhang et al., 4 May 2025, Loo et al., 4 Sep 2025, Du et al., 23 Dec 2025).