Ability of conversational agents to identify and leverage valuable information from past interactions

Determine whether Large Language Model-based conversational agents engaged in multi-session collaboration can recognize which information from past user–agent interactions will be valuable for future interactions and leverage that information effectively to improve collaboration outcomes.

Background

The paper observes that most prior memory systems and evaluations emphasize recall of past interaction details, which, while necessary, may not capture the full capability needed to improve long-term collaboration. Specifically, beyond remembering, agents must discern which pieces of information will be helpful later and use them to guide future behavior.

This uncertainty motivates the introduction of MultiSessionCollab, a benchmark that assesses whether agents can learn and apply user interaction preferences across sessions, thereby testing the broader question of recognizing and leveraging valuable information from past interactions.