Trajectory-Informed Memory Generation for Self-Improving Agent Systems

Abstract: LLM-powered agents face a persistent challenge: learning from their execution experiences to improve future performance. While agents can successfully complete many tasks, they often repeat inefficient patterns, fail to recover from similar errors, and miss opportunities to apply successful strategies from past executions. We present a novel framework for automatically extracting actionable learnings from agent execution trajectories and utilizing them to improve future performance through contextual memory retrieval. Our approach comprises four components: (1) a Trajectory Intelligence Extractor that performs semantic analysis of agent reasoning patterns, (2) a Decision Attribution Analyzer that identifies which decisions and reasoning steps led to failures, recoveries, or inefficiencies, (3) a Contextual Learning Generator that produces three types of guidance -- strategy tips from successful patterns, recovery tips from failure handling, and optimization tips from inefficient but successful executions, and (4) an Adaptive Memory Retrieval System that injects relevant learnings into agent prompts based on multi-dimensional similarity. Unlike existing memory systems that store generic conversational facts, our framework understands execution patterns, extracts structured learnings with provenance, and retrieves guidance tailored to specific task contexts. Evaluation on the AppWorld benchmark demonstrates consistent improvements, with up to 14.3 percentage point gains in scenario goal completion on held-out tasks and particularly strong benefits on complex tasks (28.5~pp scenario goal improvement, a 149\% relative increase).

• The paper presents a fully automated pipeline that extracts trajectory intelligence to generate actionable memories for improved LLM agent performance.
• The methodology includes semantic annotation, causal analysis, and adaptive retrieval, achieving significant gains in task and scenario completion rates.
• Empirical results demonstrate notable improvements on complex, long-horizon tasks, validating the approach’s robustness and practical applicability.

Trajectory-Informed Memory Generation for Self-Improving Agent Systems

Introduction

LLM-based autonomous agents confront critical limitations in leveraging prior interaction experience to drive performance improvements. The statelessness of standard LLMs inhibits systematic learning from execution trajectories—sequences of agent thoughts, actions, and feedback observed during the fulfillment of real or simulated tasks. While traditional techniques such as rule-based augmentation, prompt engineering, or vector-DB-backed conversational memory provide ad hoc benefits, they lack formal mechanisms for semantic trajectory understanding, causal analysis of decision failures or inefficiencies, contextualized strategy extraction, and provenance-aware memory retrieval.

This work introduces a fully automated, structured pipeline for trajectory-informed agent memory. The framework is driven by four core modules: (1) Trajectory Intelligence Extraction, (2) Decision Attribution Analysis, (3) Contextual Learning Generation, and (4) Adaptive Memory Retrieval. This enables LLM agents to systematically distill actionable guidance from prior execution experience, supporting on-the-fly self-improvement and reducing repeated errors, inefficient patterns, and missed strategy transfer in new task contexts.

System Design

Trajectory Intelligence Extraction

This component transforms raw agent trajectories into structured, semantically annotated representations. It classifies agent “thoughts” along functional categories such as analytical, planning, validation, and reflection, generalizing beyond explicit signals to capture meta-cognitive and self-correction patterns. It further interprets result traces and self-reflective signals to infer or contextualize outcome classifications (succeeded, failed, recovered, inefficient success).

Decision Attribution Analysis

Attribution is formalized as a multi-level causal analysis along the trajectory. The system distinguishes between failure indicators, recovery evidence, inefficiency markers, and manifestations of effective strategy. With LLM-powered semantic tracing, it identifies immediate, proximate, and root causes of failures, maps agent self-recognition patterns, and explicates the causal link between observed outcomes and underlying decisions.

Contextual Learning Generation

This module synthesizes experiences into three distinct classes:

• Strategy Tips: Extracted from clean executions, these encode high-level patterns underlying robust, effective task completion.
• Recovery Tips: Derived from failure-recovery sub-trajectories, these document diagnostic recognition, correction tactics, and sequenced remediation steps.
• Optimization Tips: Mined from inefficient yet successful trajectories, these record alternatives for improved performance or reduced redundancy.

Each tip is represented with rich metadata, explicit triggers, and negative examples to promote actionable transfer and downstream retrieval precision. Both holistic (task-level) and compositional (subtask-level) extraction are supported, enabling cross-domain generalization and phase-specific guidance.

Tip Storage, Clustering, and Consolidation

To combat redundancy and enable scalable retrieval, the system abstracts subtask descriptions via LLM-based normalization (entity abstraction, action unification), then performs semantic clustering on embeddings. An LLM-powered merging mechanism resolves redundant/contradictory entries—using success/failure metadata for conflict arbitration. This yields a curated memory system with dual representation: vector embeddings for efficient similarity search and structured attributes for contextual filtering.

Adaptive Runtime Retrieval

Two principal retrieval modes are offered:

• Cosine Similarity Retrieval: Embeds task descriptions and retrieves high-similarity tips via nearest neighbor search, using thresholds and top-k cuts to manage relevance/coverage.
• LLM-Guided Selection: Employs LLM reasoning to parse task intents, infer domains, filter by metadata, and prioritize by category/severity, supporting nuanced cross-domain matches and context-sensitive guidance.

Selected tips are injected as explicit guidelines into agent prompts prior to reasoning, supporting explicit transfer and behavioral alignment.

Empirical Results on AppWorld Benchmark

The framework is evaluated on the AppWorld benchmark, whose tasks range from straightforward API-driven goals to complex, multi-application, multi-step scenarios that stress generalization, planning, and recovery.

Key Metrics:

• Task Goal Completion (TGC): Percentage of tasks passing all end-state and programmatic tests.
• Scenario Goal Completion (SGC): Percentage of scenarios where all associated variants are passed, reflecting behavioral consistency.

Summary of Findings:

• Agents using subtask-level tips with LLM-guided retrieval achieved 73.2% TGC and 64.3% SGC on held-out tasks, compared to 69.6% TGC and 50.0% SGC for the no-memory baseline.
• The SGC improvement (+14.3~pp) is especially pronounced on Difficulty 3 tasks (+28.5~pp; 19.1%→47.6%, representing a 149% relative gain), indicating the critical value of learned strategies and recovery guidance on challenging, long-horizon control problems.
• Cosine similarity retrieval (τ=0.6, no top-k) with subtask-level tips provides slightly higher TGC (73.8%) but lags in SGC (57.1%), highlighting a tradeoff between sensitivity to individual task requirements and the consistency achievable with metadata- and category-aware retrieval.
• Task-level extraction offers strong SGC gains (62.5%) but is generally outperformed by subtask-level approaches on TGC for complex scenarios.
• On tasks recycled from the training or development partitions, performance increases further, demonstrating robust self-improvement on recurring or structurally similar challenges.

Theoretical and Practical Implications

This research marks a notable shift from ad hoc or uniform memory accumulation to structured, quality-aware, and traceable memory construction rooted in explicit causal attribution. By enabling agents to generate, consolidate, and contextually retrieve actionable procedural/declarative knowledge from operational experience, the framework fills a key gap identified in recent taxonomies of agent memory design ([zhang2024survey], [du2025rethinking]). The system substantially mitigates pitfalls such as error propagation and misaligned replay ([xiong2025memory]), offering a generalizable methodology that is model-agnostic, compositional, and naturally extensible to multi-agent and multi-domain settings.

From a deployment perspective, the approach facilitates continuous agent self-improvement without necessitating frequent manual prompt revisions or slow reward-based RL policy updates. The explicit storage of decision provenance and actionable negative examples makes downstream auditing and strategy debugging practical—essential for trust and reliability in enterprise or safety-critical deployments. Integration with platforms like IBM’s CUGA evidences industrial applicability.

Limitations and Future Directions

The current framework assumes reliable parsing of agent “thoughts,” consistent trajectory logging, and access to LLM-powered analysis at scale—potential bottlenecks in low-resource or latency-sensitive applications. While designed for modular extendibility, the pipeline’s efficacy is partially contingent on the precision and recall of the LLMs orchestrating segmentation, attribution, clustering, and consolidation. Retrieval strategies require continued tuning to balance recall with context relevance, especially in the face of prompt-length constraints or heavily compositional task intents.

Future research avenues include integrating multi-agent attribution (cross-agent strategy/meta-strategy propagation), automated selective forgetting, lifelong memory compression, and direct exploration of open-source, high-throughput LLMs ([qwen2025qwen25], [gptoss2025]) for all pipeline stages. Further, extending the analysis to embodied and multimodal settings and real-world deployment with robust online evaluation and ablation will broaden the technique’s practical reach.

Conclusion

Trajectory-informed, semantically-rich memory generation offers a scalable path toward robust, self-improving LLM agents. By extracting, consolidating, and contextually retrieving actionable guidance rooted in execution experience, agents achieve quantifiable consistency and performance gains, notably on long-horizon, multi-application tasks. This framework establishes a practical blueprint for agentic memory systems, underscoring the importance of structured learning extraction, causal attribution, and adaptive retrieval as enablers of iterative agent evolution (2603.10600).