Inconsistent illusory motion in predictive coding deep neural networks

Abstract: Why do we perceive illusory motion in some static images? Several accounts have been proposed based on eye movements, response latencies to different image elements, or interactions between image patterns and motion energy detectors. Recently, PredNet, a recurrent deep neural network (DNN) based on predictive coding principles, was reported to reproduce the "Rotating Snakes" illusion, suggesting a role for predictive coding in illusory motion. We replicate this finding and then use a series of "in silico psychophysics" experiments to examine whether PredNet behaves consistently with human observers for simplified variants of the illusory stimuli. We also measure response latencies to individual elements of the Rotating Snakes pattern by probing internal units in the network. A pretrained PredNet model predicted illusory motion for all subcomponents of the Rotating Snakes stimulus, consistent with human observers. However, we found no simple response delays in internal units, as found in physiological data. The PredNet model's detection of motion in gradients was based on contrast, not luminance, as it is in human perception. Finally, we tested the robustness of the illusion on 10 identical PredNets trained on the same video data; we found a large variation in the ability of the network to reproduce the illusion and predict motion for simplified variants of the illusion. Also, unlike human observers, none of the networks predicted illusory motion for greyscale variants of the pattern. Even when a DNN successfully reproduces some idiosyncrasy of human vision, a more detailed investigation can reveal inconsistencies between humans and the network and between different instances of the same network. The inconsistency of the Rotating Snakes illusion in PredNets trained from different initializations suggests that predictive coding does not reliably lead to human-like illusory motion.