A compressed code for memory discrimination

Abstract: The ability to discriminate similar visual stimuli is an important index of memory function. This ability is widely thought to be supported by expanding the dimensionality of relevant neural codes, such that neural representations for similar stimuli are maximally distinct, or ``separated.'' An alternative hypothesis is that discrimination is supported by lossy compression of visual inputs, efficiently coding sensory information by discarding seemingly irrelevant details. A benefit of compression, relative to expansion, is that it allows individuals to retain fewer essential dimensions underlying stimulus variation -- a process linked to higher-order visual processing -- without hindering discrimination. Under this hypothesis, pattern separation is facilitated when more information from similar stimuli can be discarded, rather than preserved. We test the compression versus expansion hypotheses by predicting performance on the canonical mnemonic similarity task. We train neural networks to compress perceptual and semantic factors of stimuli, measuring lossiness using the mathematical framework underlying compression. Consistent with the compression hypothesis, and not the expansion hypothesis, greater lossiness predicts the ease and performance of lure discrimination, especially in deeper convolutional network layers that predict higher-order visual brain activity. We then confirm these predictions across two image sets, four behavioral datasets, and alternative lossiness metrics. Finally, using task fMRI, we identify signatures of lossy compression -- neural dimensionality reduction and information loss -- in higher-order visual regions V4 and IT and hippocampal DG/CA3 and CA1 linked to lure discrimination. These results suggest lossy compression supports mnemonic discrimination by discarding redundant and overlapping information.