Balancing memory compression with performance in LLM-based agents

Determine the appropriate balance between memory compression rates and downstream task performance in memory modules for large language model–based agents, by characterizing how textual or latent memory extraction and compression affect accuracy and cost and by identifying strategies that retain salient information while minimizing token usage and inference latency.

Background

The survey highlights that memory extraction and compression are central to improving agent efficiency by reducing token consumption and context-window saturation. However, aggressive compression can discard critical information and degrade task accuracy. Empirical results cited for systems such as LightMem show a clear trade-off: higher compression lowers cost but can harm performance, whereas milder compression preserves accuracy at greater cost.

AgentFold and other memory-centric methods demonstrate the potential of proactive context management and summarization, yet the precise compression level that optimally balances efficiency (tokens, latency) and effectiveness (task success, reasoning quality) is not established. The authors call for a principled understanding of this trade-off and methods that preserve key signals during compression.