When Adaptive Diffusion Algorithm Converges to True Parameter?

Abstract: We attempt to answer the question what data brings adaptive diffusion algorithms converging to true parameters. The discussion begins with the diffusion recursive least squares (RLS). When unknown parameters are scalar, the necessary and sufficient condition of the convergence for the diffusion RLS is established, in terms of the strong consistency and mean-square convergence both. However, for the general high dimensional parameter case, our results suggest that the diffusion RLS in a connected network might cause a diverging error, even if local data at every node could guarantee the individual RLS tending to true parameters. Due to the possible failure of the diffusion RLS, we prove that the diffusion Robbins-Monro (RM) algorithm could achieve the strong consistency and mean-square convergence simultaneously, under some cooperative information conditions. The convergence rates of the diffusion RM are derived explicitly.