Hierarchical Task Memory

• Hierarchical task memory is a structured architecture that segregates high-level planning from fine-grained execution to capture multi-agent experiences.
• It improves generalization and knowledge transfer by leveraging dual-memory systems and layered reflection mechanisms in long-horizon tasks.
• Advanced designs use pointer routing and dynamic memory updates to optimize retrieval speed, scalability, and interpretability in agent systems.

Hierarchical task memory refers to structured memory architectures and mechanisms that capture, store, retrieve, and utilize agent or multi-agent experiences at multiple levels of abstraction, typically separating high-level planning representations from fine-grained execution traces. These approaches underpin efficient knowledge transfer, compositional decision-making, and robust generalization in long-horizon, multi-task, and multi-agent LLM-based systems. Recent research establishes the superiority of hierarchical task memory over flat or monolithic episodic memory for generalization, computational efficiency, and interpretability in LLM agents and agentic systems.

1. Formal Structures for Hierarchical Task Memory

A hierarchical task memory architecture partitions agent experience into distinct, semantically organized layers supporting both abstraction and specificity. A representative two-level instantiation is the dual-memory structure in H$^2$R, which introduces:

• High-level planning memory ($M_H$): Each unit $m_H^i = (k_H^i,v_H^i)$ encodes task-level information. Keys $k_H^i$ are dense embeddings of task descriptions, and values $v_H^i$ include the task text $X^i$, successful subgoal sequence $G_+^i = \langle g_1^i, ..., g_{K_i}^i \rangle$, and distilled planning insights $I_\text{high}^i$.
• Low-level execution memory ($M_L$): Each unit $m_L^j = (k_L^j,v_L^j)$ encodes subgoal execution details. Keys $k_L^j$ are embeddings of subgoal texts $g^j$, and values $v_L^j$ include the subgoal $g^j$, execution sub-trajectory $\tau^j = \langle (a_t,o_t) \rangle$, and execution insights $I_\text{low}^j$.

Memory management involves continual appending upon new experience. When exceeding capacity ($C_H$, $C_L$), the least-useful units (determined by vote counts in their insights) are evicted (Ye et al., 16 Sep 2025).

More complex stratifications appear in H-MEM, which organizes memory into four levels: domain, category, memory trace, and episode. Each entry stores a semantic embedding, self index, and child indices, forming a multi-layer pointer network supporting top-down retrieval (Sun et al., 23 Jul 2025).

In multi-agent contexts, G-Memory formalizes experience as a three-tier graph: the interaction graph (utterance-level), the query graph (task-level), and the insight graph (distilled lessons), enabling bi-directional navigation between high-level abstractions and raw execution data (Zhang et al., 9 Jun 2025).

2. Construction, Distillation, and Update Mechanisms

Hierarchical task memories require mechanisms to distill experience into reusable abstractions at each level. The H$^2$R system centers around the Hierarchical Hindsight Reflection (H$^2$R) algorithm, which alternates between:

• High-level reflection: Processes full task trajectories, segmenting them into minimal subgoal sequences and extracting planning insights by comparing successful and unsuccessful experiences. Planning insights are updated via add/modify/upvote/downvote operations.
• Low-level reflection: Decomposes successful task trajectories by subgoal, associating each with a sub-trajectory and extracting fine-grained execution insights via contrastive analysis.
• Grounding: Associates globally discovered insights to specific memory units through relevancy filtering.

Update strategy appends new memory units, evicts underused units, and maintains fixed-size, ranked sets of rules for transferability. Reflection is computationally bounded via insight set size limits (Ye et al., 16 Sep 2025).

Advanced architectures such as StackPlanner implement explicit memory revision operations—condensation (summarization), pruning (removal of low-utility segments), and experience memory update—triggered by stack overflow or error detection, optimizing context length and minimizing error propagation (Zhang et al., 9 Jan 2026).

Cognitive-inspired symbolic frameworks, such as those using fuzzy Description Logic, implement "store," "retrieve," "consolidate," and "forget" via score-driven heuristics, supporting one-shot learning, scene abstraction, and dynamic restructuring of the concept hierarchy (Buoncompagni et al., 2024).

3. Retrieval and Utilization in Inference

Hierarchical retrieval mechanisms provide targeted, efficient access to relevant abstractions:

• High-level retrieval: Given a new task, its embedding is compared against keys in $M_H$ via cosine similarity, retrieving the top-k most similar planning memories. Used for subgoal generation and strategic planning.
• Low-level retrieval: Each subgoal emitted by the planner is mapped against $M_L$ to fetch corresponding execution trajectories and insights.
• Prompt construction: Retrieved units seed the LLM's context as structured in-context examples, ensuring alignment with the task's compositional structure (Ye et al., 16 Sep 2025).

H-MEM employs a layer-by-layer index routing: a query embedding traverses from high-level semantic categories down to episode memories, at each layer selecting top-k candidates given stored index-pointers, dramatically reducing retrieval complexity versus flat memory (Sun et al., 23 Jul 2025).

In multi-agent systems, role-specific prompt construction uses bi-directional graph traversal to assemble both strategic insights (from the insight graph) and fine-grained collaboration trajectories (from the interaction graph), filtered according to current agent role (Zhang et al., 9 Jun 2025).

Frameworks such as Task Memory Engine (TME) dynamically synthesize LLM prompts using only the active path in a task memory tree, allowing for token-efficient, coherent, and interpretable prompting in hierarchical, multi-step tasks (Ye, 11 Apr 2025).

4. Empirical Findings and Comparative Outcomes

Empirical results across multiple domains confirm that hierarchical task memory architectures yield substantial improvements over flat or linear memory systems:

Algorithm	AlfWorld	PDDLGame	2Wiki (F1)	GQA (%)
ReAct (no memory)	46.3	66.7	—	60.2
Expel/flat Episodic	72.4	72.2	—	64.6
H$^2$R (hierarchical)	75.9	80.5	—	68.4

Ablation studies demonstrate that removing either planning or execution memory from H$^2$R causes catastrophic performance drops (e.g., PDDLGame: $-27.7$pp/ $-19.4$pp), confirming the indispensability of both abstraction levels (Ye et al., 16 Sep 2025).

H-MEM reduces retrieval time from >400ms (MemoryBank) to <100ms under large-scale memory, with average F1 gains up to +14.7 points and substantial boosts on multi-hop QA and adversarial reasoning (Sun et al., 23 Jul 2025).

In multi-agent coordination and embodied scenarios, G-Memory and MiTa boost success rates or efficiency by 10–20 points, prevent behavioral conflict, and ensure consistent task distribution by grounding local agent processes in global, multi-level context (Zhang et al., 9 Jun 2025, Zhang et al., 30 Jan 2026).

ReAcTree's composition of episodic and working memory at subgoal and environment levels results in +30pp goal success rate improvement on long-horizon planning over strong baselines (Choi et al., 4 Nov 2025).

5. Variants in Multi-Agent and Symbolic Systems

Hierarchical task memory principles appear across a spectrum of agentic system designs, including:

• Centralized–decentralized hybrid systems: StackPlanner and MiTa employ a top-level manager/central coordinator with episodic and plan memory, while sub-agents maintain ephemeral execution/local memory, ensuring long-horizon coherence without overloading central context (Zhang et al., 9 Jan 2026, Zhang et al., 30 Jan 2026).
• Graph-structured and DAG-aware memory: Task Memory Engine and G-Memory generalize beyond trees to DAGs and interlinked graphs, supporting reusable subtasks, cross-task insight transfer, and dependency constraints (Ye, 11 Apr 2025, Zhang et al., 9 Jun 2025).
• Symbolic and fuzzy-logic-based memory: Hierarchically-structured, score-driven fuzzy ontologies enable cognitively-inspired robots to dynamically build and prune scene/concept hierarchies based on structural similarity, frequency, and consolidation heuristics (Buoncompagni et al., 2024).

These extensions retain the core principle of leveraging structural decomposability for improved reasoning, reuse, and efficient memory pruning.

6. Design Recommendations and Best Practices

Empirical and algorithmic analyses yield design guidelines for constructing effective hierarchical task memories:

• Limit prompt retrievals to a small $k$ at each level to ensure context diversity and computational tractability (e.g., top-3 or top-10) (Ye et al., 16 Sep 2025, Sun et al., 23 Jul 2025).
• Optimize embedding dimensionality ($D$) for semantic fidelity; values around 512–1024 are typical (Sun et al., 23 Jul 2025, Ye et al., 16 Sep 2025).
• Maintain separate insight/rule sets at each level and upvote/downvote based on utility in subsequent tasks to keep memory live and dynamically adapted (Ye et al., 16 Sep 2025, Qiao et al., 28 May 2025).
• Combine pointer-based routing, automatic forgetting, and user/feedback-based updating for robust, self-renewing long-term memory (Sun et al., 23 Jul 2025).
• For tree/graph memory, prune completed or irrelevant subtrees and monitor token growth for summarization or checkpointing (Ye, 11 Apr 2025).

7. Impact and Limitations

Hierarchical task memory systems underlie significant recent breakthroughs in agent generalization, compositional planning, and multi-agent collaboration. They address both scalability (memory and compute), control compositionality, and interpretability demands.

Limitations include the need for careful hyperparameter tuning (e.g., insight set sizes, prompt budgets), potential reasoning bottlenecks in symbolic/FOL systems, and open questions regarding optimal depth/breadth tradeoffs for arbitrary domains (Sun et al., 23 Jul 2025, Buoncompagni et al., 2024). Future directions point towards adaptive layer depth, stronger transfer across task graphs, and automated summarization mechanisms.

---

References:

• H$^2$R: Hierarchical Hindsight Reflection for Multi-Task LLM Agents (Ye et al., 16 Sep 2025)
• Hierarchical Memory for High-Efficiency Long-Term Reasoning in LLM Agents (Sun et al., 23 Jul 2025)
• StackPlanner: A Centralized Hierarchical Multi-Agent System with Task-Experience Memory Management (Zhang et al., 9 Jan 2026)
• MiTa: A Hierarchical Multi-Agent Collaboration Framework with Memory-integrated and Task Allocation (Zhang et al., 30 Jan 2026)
• G-Memory: Tracing Hierarchical Memory for Multi-Agent Systems (Zhang et al., 9 Jun 2025)
• ReAcTree: Hierarchical LLM Agent Trees with Control Flow for Long-Horizon Task Planning (Choi et al., 4 Nov 2025)
• Task Memory Engine (TME): A Structured Memory Framework with Graph-Aware Extensions for Multi-Step LLM Agent Tasks (Ye, 11 Apr 2025)
• Efficiently Enhancing General Agents With Hierarchical-categorical Memory (Qiao et al., 28 May 2025)
• Evolving Hierarchical Memory-Prediction Machines in Multi-Task Reinforcement Learning (Kelly et al., 2021)
• Learning Symbolic Task Representation from a Human-Led Demonstration: A Memory to Store, Retrieve, Consolidate, and Forget Experiences (Buoncompagni et al., 2024)
• Cognitive Approach to Hierarchical Task Selection for Human-Robot Interaction in Dynamic Environments (Bukhari et al., 2023)