Stochastic processes with random contexts: a characterization, and adaptive estimators for the transition probabilities

Abstract: This paper introduces the concept of random context representations for the transition probabilities of a finite-alphabet stochastic process. Processes with these representations generalize context tree processes (a.k.a. variable length Markov chains), and are proven to coincide with processes whose transition probabilities are almost surely continuous functions of the (infinite) past. This is similar to a classical result by Kalikow about continuous transition probabilities. Existence and uniqueness of a minimal random context representation are proven, and an estimator of the transition probabilities based on this representation is shown to have very good "pastwise adaptativity" properties. In particular, it achieves minimax performance, up to logarithmic factors, for binary renewal processes with bounded $2+\gamma$ moments.