Credit Assignment for Long-Horizon Agentic Reasoning

Develop principled and generalizable credit-assignment algorithms for long-horizon large language model-based agentic systems that integrate token-level decisions, external tool invocations, skill selection, and memory operations, and enable learning that transfers across extended sequences of episodes and tasks.

Background

The survey highlights that while methods such as ReAct and Tree-of-Thought improve short-horizon reasoning, agents still accumulate errors over long tasks. Reinforcement-learning agents like WebRL and Agent-R1 rely on domain-specific rewards and largely treat episodes independently, which limits generalization.

Process-aware approaches attempt finer-grained credit signals but remain environment-specific. A key gap is a unified way to attribute success or failure across heterogeneous decision elements (tokens, tool calls, skills, memory updates) and to generalize such learning across tasks and episodes.