Learning in Random Utility Models Via Online Decision Problems

Abstract: This paper studies the Random Utility Model (RUM) in a repeated stochastic choice situation, in which the decision maker is imperfectly informed about the payoffs of each available alternative. We develop a gradient-based learning algorithm by embedding the RUM into an online decision problem. We show that a large class of RUMs are Hannan consistent (\citet{Hahn1957}); that is, the average difference between the expected payoffs generated by a RUM and that of the best-fixed policy in hindsight goes to zero as the number of periods increase. In addition, we show that our gradient-based algorithm is equivalent to the Follow the Regularized Leader (FTRL) algorithm, which is widely used in the machine learning literature to model learning in repeated stochastic choice problems. Thus, we provide an economically grounded optimization framework to the FTRL algorithm. Finally, we apply our framework to study recency bias, no-regret learning in normal form games, and prediction markets.