MIRIX: Multi-Agent Memory System for LLM-Based Agents

Abstract: Although memory capabilities of AI agents are gaining increasing attention, existing solutions remain fundamentally limited. Most rely on flat, narrowly scoped memory components, constraining their ability to personalize, abstract, and reliably recall user-specific information over time. To this end, we introduce MIRIX, a modular, multi-agent memory system that redefines the future of AI memory by solving the field's most critical challenge: enabling LLMs to truly remember. Unlike prior approaches, MIRIX transcends text to embrace rich visual and multimodal experiences, making memory genuinely useful in real-world scenarios. MIRIX consists of six distinct, carefully structured memory types: Core, Episodic, Semantic, Procedural, Resource Memory, and Knowledge Vault, coupled with a multi-agent framework that dynamically controls and coordinates updates and retrieval. This design enables agents to persist, reason over, and accurately retrieve diverse, long-term user data at scale. We validate MIRIX in two demanding settings. First, on ScreenshotVQA, a challenging multimodal benchmark comprising nearly 20,000 high-resolution computer screenshots per sequence, requiring deep contextual understanding and where no existing memory systems can be applied, MIRIX achieves 35% higher accuracy than the RAG baseline while reducing storage requirements by 99.9%. Second, on LOCOMO, a long-form conversation benchmark with single-modal textual input, MIRIX attains state-of-the-art performance of 85.4%, far surpassing existing baselines. These results show that MIRIX sets a new performance standard for memory-augmented LLM agents. To allow users to experience our memory system, we provide a packaged application powered by MIRIX. It monitors the screen in real time, builds a personalized memory base, and offers intuitive visualization and secure local storage to ensure privacy.

• The paper presents MIRIX's main contribution by introducing a multi-agent memory system with six specialized memory types to enhance context and personalization.
• It details a flexible architecture that improved accuracy by 35% in ScreenshotVQA and reached 85.4% in long-form conversational tasks.
• The system’s modular design enables efficient multimodal data management, reducing storage needs by 99.9% compared to traditional approaches.

MIRIX: Multi-Agent Memory System for LLM-Based Agents

Introduction

"MIRIX: Multi-Agent Memory System for LLM-Based Agents" presents an innovative approach to enhancing memory capabilities in AI systems, particularly focusing on LLM agents. The paper introduces MIRIX, a comprehensive memory architecture designed to address limitations of existing systems that rely heavily on flat and static memory structures. MIRIX comprises six specialized memory components: Core, Episodic, Semantic, Procedural, Resource Memories, and a Knowledge Vault, each managed by dedicated agents for efficient information retrieval and management. This system significantly improves the ability of AI agents to remember, contextualize, and personalize interactions over time by embracing multimodal data and a multi-agent framework.

Figure 1: The six memory components of MIRIX, each providing specialized functionality.

Memory Architecture and Components

MIRIX's architecture is devised to provide a flexible, modular solution to memory management in LLM agents. The six memory types serve distinct roles within the system:

• Core Memory stores persistent personalized data about the user and agent profiles, essential for continuous personalized interactions.
• Episodic Memory retains time-stamped user-specific events, supporting temporal reasoning and context tracking.
• Semantic Memory encompasses general knowledge and the social graph of the user, essential for abstract reasoning and understanding relationships.
• Procedural Memory encapsulates actionable knowledge in workflows or scripts that aid users in complex tasks.
• Resource Memory manages documents, transcripts, or files to maintain context continuity in extended engagements.
• Knowledge Vault is reserved for sensitive, verbatim information required for secured tasks (e.g., API keys).

These components are interconnected via a Multi-Agent System, where each memory type is managed by a specialized agent overseen by a Meta Memory Manager. This structure enables efficient routing and retrieval, enhancing the agent's ability to learn and evolve from continual interactions with real-world data.

Implementation and Use Cases

Implementation of MIRIX is demonstrated through two primary benchmarks: ScreenshotVQA and LOCOMO. In ScreenshotVQA, MIRIX processes a vast amount of high-resolution screenshot data from users' computers, achieving a 35% increase in accuracy and a 99.9% reduction in storage requirements compared with RAG baselines.

In the LOCOMO dataset—focusing on long-form, multi-turn conversations—MIRIX surpasses existing models with an accuracy of 85.4%, marking a significant improvement from prior state-of-the-art systems. This exemplifies MIRIX's proficiency in managing extensive conversational data and its potential to deliver highly personalized, long-term memory capabilities.

MIRIX also finds practical application in a developed cross-platform personal assistant. This application, built with React-Electron, actively monitors user screen activity to dynamically update its memory, facilitating informed user queries and interactions.

Figure 2: Chat Window

Evaluation Results

Extensive evaluation demonstrated the effectiveness of MIRIX across different tasks. In LOCOMO, tasks were categorized into single-hop, multi-hop, temporal, and open-domain questions. MIRIX showed exceptional performance, particularly excelling at temporal and multi-hop reasoning, surpassing competitive baselines by a substantial margin.

The performance illustrates MIRIX's advantage in integrating comprehensive memory types with sophisticated retrieval strategies, resulting in more accurate and contextually aware outputs. This success underscores the potential of integrating nuanced, structured memory models in LLM agents to enhance utility and user experience.

Conclusion

MIRIX sets a new standard for memory-augmented agents by combining a rich, multi-component memory architecture with a powerful multi-agent framework. Its ability to process and abstract multimodal information significantly enhances an agent's functionality in real-world scenarios. The advancements presented in MIRIX pave the way for future developments in AI systems that require dynamic, scalable, and efficient memory management. The system exemplifies a crucial step toward intelligent agents capable of true memory persistence and contextual reasoning, critical for advanced human-AI interaction. Future work may explore the integration of MIRIX into wearable technology, expanding its accessibility and adoption.