Effect of observation history on longer-horizon tasks

Determine how incorporating observation history affects the performance of LLM-based web agents on tasks with horizons longer than 15 steps, and characterize how performance scales with history length in such settings.

Background

The reported experiments investigate observation history using accessibility-tree inputs on WorkArena L1 tasks, which are at most 15 steps long, and show that history generally improves success rates.

The authors explicitly state that the impact of observation history on longer-horizon tasks is unknown, leaving open how benefits scale as task length increases.