Decision-making with distorted memory: Escaping the trap of past experience

Abstract: Snapshots of "best" (or "worst") experience are known to dominate human memory and may thus also have a significant effect on future behaviour. We consider here a model of repeated decision-making where, at every time step, an agent takes one of two choices with probabilities which are functions of the maximum utilities previously experienced. Depending on the utility distributions and the level of noise in the decision process, it is possible for an agent to become "trapped" in one of the choices on the basis of their early experiences. If the utility distributions for the two choices are different, then the agent may even become trapped in the choice which is objectively worse in the sense of expected long-term returns; crucially we extend earlier work to address this case. Using tools from statistical physics and extreme-value theory, we show that for exponential utilities there is an optimal value of noise which maximizes the expected returns in the long run. We also briefly discuss the behaviour for other utility distributions.