Visual Episodic Memory-based Exploration
Abstract: In humans, intrinsic motivation is an important mechanism for open-ended cognitive development; in robots, it has been shown to be valuable for exploration. An important aspect of human cognitive development is $\textit{episodic memory}$ which enables both the recollection of events from the past and the projection of subjective future. This paper explores the use of visual episodic memory as a source of intrinsic motivation for robotic exploration problems. Using a convolutional recurrent neural network autoencoder, the agent learns an efficient representation for spatiotemporal features such that accurate sequence prediction can only happen once spatiotemporal features have been learned. Structural similarity between ground truth and autoencoder generated images is used as an intrinsic motivation signal to guide exploration. Our proposed episodic memory model also implicitly accounts for the agent's actions, motivating the robot to seek new interactive experiences rather than just areas that are visually dissimilar. When guiding robotic exploration, our proposed method outperforms the Curiosity-driven Variational Autoencoder (CVAE) at finding dynamic anomalies.
- 2016. Unifying count-based exploration and intrinsic motivation. In Advances in Neural Information Processing Systems.
- Brooks, R. 1986. A robust layered control system for a mobile robot. IEEE Journal on Robotics and Automation 2(1):14–23.
- 1977. Flashbulb memories. Cognition 5(1):73–99.
- 2018a. Large-scale study of curiosity-driven learning. arXiv preprint arXiv:1808.04355.
- 2018b. Exploration by random network distillation. arXiv preprint arXiv:1810.12894.
- 2019. Learning exploration policies for navigation. In International Conference on Learning Representations.
- 2017. Abnormal event detection in videos using spatiotemporal autoencoder. In International Symposium on Neural Networks, 189–196. Springer.
- 2006. Simultaneous localization and mapping. IEEE Robotics and Automation Magazine 13(2):99–110.
- 2012. Active inference and agency: optimal control without cost functions. Biological Cybernetics 106(8):523–541.
- Gardiner, J. 2008. Remembering and knowing. In Byrne, J. H., ed., Learning and Memory: A Comprehensive Reference. Oxford: Academic Press. 285–305.
- 1990. An integrated navigation and motion control system for autonomous multisensory mobile robots. In Autonomous robot vehicles. Springer. 420–443.
- 2020. Curiosity-driven variational autoencoder for deep q network. In Pacific-Asia Conference on Knowledge Discovery and Data Mining, 764–775. Springer.
- 2016. Learning temporal regularity in video sequences. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 733–742.
- Hörner, J. 2016. Map-merging for multi-robot system. Bachelor’s thesis, Charles University in Prague, Faculty of Mathematics and Physics, Prague.
- 2017. Overcoming catastrophic forgetting in neural networks. Proceedings of the National Academy of Sciences 114(13):3521–3526.
- 1991. A robot exploration and mapping strategy based on a semantic hierarchy of spatial representations. Robotics and Autonomous Systems 8(1-2):47–63.
- Lin, Y.-T. 2018. Visual perspectives in episodic memory and the sense of self. Frontiers in Psychology 9:2196.
- 2017. Count-based exploration with neural density models. In Proceedings of the International Conference on Machine Learning.
- 2007. What is intrinsic motivation? a typology of computational approaches. Frontiers in Neurorobotics 1.
- 2017. Curiosity-driven exploration by self-supervised prediction. In International Conference on Machine Learning, 2778–2787. PMLR.
- 2018. Sidekick policy learning for active visual exploration. In Proceedings of the European Conference on Computer Vision (ECCV), 413–430.
- 2021. An exploration of embodied visual exploration. International Journal of Computer Vision 129(5):1616–1649.
- 2018. Deep episodic memory: Encoding, recalling, and predicting episodic experiences for robot action execution. IEEE Robotics and Automation Letters 3.
- 2018. Episodic curiosity through reachability. arXiv preprint arXiv:1810.02274.
- 2018. Properties of visual episodic memory following repeated encounters with objects. Learning and Memory 25:309––316.
- 2015. Convolutional LSTM network: A machine learning approach for precipitation nowcasting. In Cortes, C.; Lawrence, N. D.; Lee, D. D.; Sugiyama, M.; and Garnett, R., eds., Neural Information Processing Systems, 802–810.
- 2018. An empirical study of example forgetting during deep neural network learning. arXiv preprint arXiv:1812.05159.
- Tulving, E. 1972. Episodic and semantic memory. Organization of Memory 381–403.
- Tulving, E. 2002. Episodic memory: from mind to brain. Annual Review of Psychology 53:1–25.
- 1997. Toward a theory of episodic memory: the frontal lobes and autonoetic consciousness. Psychology Bulletin 121(3):331–354.
- Yamauchi, B. 1997. A frontier-based approach for autonomous exploration. In Proceedings of the International Symposium on Computational Intelligence at ICRA, 146–151. IEEE.
- 2018. Curiosity-driven exploration for mapless navigation with deep reinforcement learning. arXiv preprint arXiv:1804.00456.
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
Top Community Prompts
Collections
Sign up for free to add this paper to one or more collections.