Imprint: Online Memory Compression for Long-Horizon Egocentric QA

Abstract: Long-horizon egocentric question answering involves answering about events that have occurred hours or days in the past. This requires memory representations that remain both retrieval-effective and scalable over days or weeks of recording. Existing long-horizon egocentric QA methods construct memory as hierarchical textual summaries of observations. While effective for reducing memory size, summarization optimizes for descriptive compression rather than retrieval: repeated interactions are absorbed into coarse textual descriptions instead of being preserved as explicit, recurring memory units, making long-horizon evidence aggregation difficult. We propose Imprint, an interaction-centric memory framework that formulates long-horizon egocentric memory as an online memory compression problem rather than summarization. Incoming observations are first represented as structured Interaction Records and continuously organized into recurring interaction patterns. Using human memory consolidation signals of recurrence, recency, and distinctiveness, Imprint selectively retains and compresses interactions into a compact retrieval-oriented memory. We evaluate Imprint on EgoLifeQA, a seven-day egocentric benchmark containing questions that require reasoning over interactions occurring hours to days before the query. With the same LLM, Imprint improves QA accuracy from 31.0% to 35.8%, increases evidence-grounded answers by $6\times$ compared with EgoRAG, reduces memory footprint by $2.3\times$, and decreases retrieval latency by $11.8\times$. These results demonstrate that memory compression provides a scalable and retrieval-effective foundation for long-horizon egocentric question answering.

• The paper presents an online memory compression pipeline that structures egocentric interactions using event prototypes, importance scoring, and consolidation for improved QA.
• It achieves a 6× increase in evidence grounding and reduces memory size and latency by 2.3× and 11.8× compared to caption-based methods.
• The framework supports scalable, long-range behavioral analytics with robust accuracy on multi-day queries in continuous lifelogging scenarios.

Imprint: Online Memory Compression for Long-Horizon Egocentric QA

Problem Overview and Motivation

Long-horizon egocentric question answering (QA) demands scalable and retrieval-effective memory representations to answer questions about personal experience distributed across hours or days, as captured by wearable cameras in continuous lifelogging scenarios. Existing approaches, such as hierarchical caption-based memory frameworks (e.g., EgoRAG), reduce storage by summarizing observations but compress away critical recurring interactions and their temporal structure, undermining long-range aggregation necessary for behavioral reasoning and longitudinal analytics. Specifically, repeated person–action–object interactions are typically absorbed into coarse summaries, making retrieval and evidence aggregation for temporally dispersed queries unreliable.

Imprint is introduced to reframe egocentric memory for QA as an online, interaction-centric memory compression task, targeting scalability and evidence-grounded retrieval for multi-day reasoning tasks.

Figure 1: Explicit preservation and aggregation of interaction structure by Imprint facilitate correct behavioral trends (e.g., meal patterns) across days, in contrast to caption-based approaches limited to within-day evidence.

Imprint Framework: Structured Online Memory Compression

Imprint’s memory system is built upon structured Interaction Records (IRs), where every LLM-processed observation is represented as a tuple: $$\langle \text{person}, \text{action}, \text{object}, t^s, t^e \rangle$$, retaining high-fidelity information necessary for long-range temporal reasoning. This formulation is in contrast to free-form or hierarchical textual summaries.

Three-Stage Online Compression Pipeline

Imprint’s memory formation is executed online through three stages:

1. Event-Prototype Assignment: Incoming IRs are mapped to event prototypes in a dynamic prototype bank based on contextualized action-object similarity, incrementally updating recurrence statistics. A thresholded similarity measure retains representational granularity while controlling memory growth.
2. Interaction Importance Scoring: Each IR is assigned an importance score, computed as $$w(f) = \log(1 + \nu) \cdot e^{-\lambda \Delta T} \cdot \left[1/(1+\log(1+N))\right]$$, incorporating cognitive signals inspired by human consolidation—recurrence, recency, and distinctiveness.
3. Interaction Consolidation: High-importance, redundant IRs are merged into temporally ordered compact entries, forming the compressed memory $\mathcal{M}$, enabling explicit behavioral trend retrieval and longitudinal evidence aggregation.

   Figure 2: The Imprint pipeline progressively compresses incoming LLM-parsed interaction records into an efficient, retrieval-oriented memory with structured recurrence and context statistics.

Retrieval and Question Answering

At query time, Imprint retrieves relevant evidence from $\mathcal{M}$ by matching question-decomposed action/object cues and temporal constraints against stored IRs. Retrieval scoring jointly considers entity similarity, action similarity, and memorized importance, yielding top-$k$ relevant entries, which are filtered by temporal logic where required. The selected evidence is then supplied to a LLM (here, Qwen2.5-7B-Instruct) for answer synthesis. This design explicitly supports queries demanding evidence over variable temporal spans, with strong grounding.

Experimental Validation

Experiments are conducted on EgoLifeQA, a seven-day continuous egocentric video QA benchmark comprising 3,000 questions of diverse temporal scope and inference type. Benchmark categories include EntityLog, EventRecall, RelationMap, HabitInsight, and TaskMaster.

Key findings:

• QA Accuracy Improvement: Imprint outperforms the reproduced EgoRAG baseline by a margin (35.8% vs. 31.0% using identical LLM and captioning pipelines).
• Evidence Grounding: Grounded answer rate increases 6× (64.8% vs. 10.8%), indicating that Imprint retrieves and aligns with explicit evidence rather than relying on LLM priors.
• Scalability: Compressed memory size is reduced by 2.3× and retrieval latency by 11.8× relative to EgoRAG, demonstrating practical viability for long-horizon deployment.
• Long-Range Robustness: Grounded accuracy remains stable (53–64%) across queries requiring evidence up to 24 hours old; summary-based frameworks display severe degradation.
• Category-Wise Gains: Largest improvements are seen in categories requiring extraction and aggregation of distributed interactions (EntityLog, EventRecall, RelationMap, HabitInsight).

  Figure 3: Imprint yields lower retrieval latency and memory usage, while maintaining high grounded accuracy even as the temporal gap between query and evidence grows.

  

  Figure 4: Imprint’s memory growth is sublinear relative to EgoRAG, reflecting effective consolidation of recurring patterns across days.

  

  Figure 5: For event recall, Imprint retrieves temporally precise and contextually complete evidence, supporting grounded answers; caption-based retrieval methods fail to recover the critical interaction.

Ablation Studies and Analysis

Ablation experiments dissect the contribution of event-prototype assignment, importance weighting, and consolidation:

• Removing event-prototype assignment notably reduces performance, confirming the necessity of recurring pattern abstraction for robust memory consolidation.
• Each cognitive importance signal—frequency, recency, distinctiveness—contributes to retrieval quality, with frequency being the most influential.
• Hyperparameter studies reveal that moderate context-history weighting and recency decay coefficients optimize both precision and recall.
• Top-K retrieval ablation and retrieval scoring analysis further demonstrate the centrality of action similarity and the necessity of balancing recall with distractor suppression.

Qualitative Evaluation

Qualitative scenarios illustrate Imprint’s advantage in:

• Temporal disambiguation for events with contradictory evidence (Figure 6)
• Accurate retrieval of co-action evidence (Figure 7)
• Counteracting object-centric retrieval bias in complex activity sequences (Figure 8)

Each case demonstrates improved evidence alignment between query intent and the underlying episodic data.

Implications and Future Directions

Imprint establishes that long-horizon egocentric QA benefits from structured, interaction-centric, cognitively inspired online memory compression, as opposed to undifferentiated textual summarization. The framework’s explicit handling of recurrence, recency, and distinctiveness supports robust evidence aggregation over multi-day trajectories, which is critical for behavior and habit analysis in personal and assistive AI agents.

Practically, Imprint enables more efficient and interpretable deployment on constrained hardware and supports evidence-grounded explanations—key for personal analytics, cognitive prosthetics, and continuous life logging systems.

Theoretically, Imprint’s cognitive heuristics could be further augmented with adaptively learned consolidation and retrieval policies, leveraging reinforcement or meta-learning to tune memory organization dynamically in response to retrieval utility and user goals.

Conclusion

Imprint provides a principled, cognitively motivated solution to scalable, evidence-grounded memory for long-horizon egocentric QA. Through online prototype-driven compression and interaction-level structuring, it achieves higher retrieval accuracy, greater evidence grounding, and superior computational efficiency compared to existing caption-based methods. This work highlights the central importance of explicit interaction modeling and selective consolidation for next-generation AI memory systems handling persistent, temporally extended data (2607.00696).