Note on the (non-)smoothness of discrete time value functions

Abstract: We consider the discrete time stopping problem [ V(t,x) = \sup_{\tau}E_{(t,x)}[g(\tau, X_\tau)],] where $X$ is a random walk. It is well known that the value function $V$ is in general not smooth on the boundary of the continuation set $\partial C$. We show that under some conditions $V$ is not smooth in the interior of $C$ either. More precisely we show that $V$ is not differentiable in the $x$ component on a dense subset of $C$. As an example we consider the Chow-Robbins game. We give evidence that as well $\partial C$ is not smooth and that $C$ is not convex, even if $g(t,\cdot)$ is for every $t$.