Null Space Properties of Neural Networks with Applications to Image Steganography
Abstract: This paper explores the null space properties of neural networks. We extend the null space definition from linear to nonlinear maps and discuss the presence of a null space in neural networks. The null space of a given neural network can tell us the part of the input data that makes no contribution to the final prediction so that we can use it to trick the neural network. This reveals an inherent weakness in neural networks that can be exploited. One application described here leads to a method of image steganography. Through experiments on image datasets such as MNIST, we show that we can use null space components to force the neural network to choose a selected hidden image class, even though the overall image can be made to look like a completely different image. We conclude by showing comparisons between what a human viewer would see, and the part of the image that the neural network is actually using to make predictions and, hence, show that what the neural network ``sees'' is completely different than what we would expect.
- “Threat of Adversarial Attacks on Deep Learning in Computer Vision: A Survey” In IEEE Access 6, 2018, pp. 14410–14430 DOI: 10.1109/ACCESS.2018.2807385
- Philip Ball “Is AI leading to a reproducibility crisis in science?” In Nature 624.7990 Nature, 2023, pp. 22–25
- Shumeet Baluja “Hiding images within images” In IEEE transactions on pattern analysis and machine intelligence 42.7 IEEE, 2019, pp. 1685–1697
- “EMNIST: Extending MNIST to handwritten letters” In 2017 International Joint Conference on Neural Networks (IJCNN), 2017 DOI: 10.1109/ijcnn.2017.7966217
- Matthew Cook, Alina Zare and Paul Gader “Outlier Detection through Null Space Analysis of Neural Networks”, 2020 arXiv:2007.01263 [cs.LG]
- “Reversible image steganography scheme based on a U-Net structure” In IEEE Access 7 IEEE, 2019, pp. 9314–9323
- Ian J Goodfellow, Jonathon Shlens and Christian Szegedy “Explaining and harnessing adversarial examples”, 2015 arXiv:1412.6572 [stat.ML]
- “Adversarial examples are not bugs, they are features” In Advances in neural information processing systems 32, 2019
- Alex Krizhevsky “Learning multiple layers of features from tiny images”, 2009
- Yann LeCun, Corinna Cortes and CJ Burges “MNIST handwritten digit database” In ATT Labs [Online]. Available: http://yann.lecun.com/exdb/mnist 2, 2010
- “Universal adversarial perturbations” In Proceedings of the IEEE conference on computer vision and pattern recognition, 2017, pp. 1765–1773
- “Quantifying Overfitting: Evaluating Neural Network Performance through Analysis of Null Space”, 2023 arXiv:2305.19424 [cs.LG]
- Jiawei Su, Danilo Vasconcellos Vargas and Kouichi Sakurai “One pixel attack for fooling deep neural networks” In IEEE Transactions on Evolutionary Computation 23.5 IEEE, 2019, pp. 828–841
- “Image steganography: A review of the recent advances” In IEEE access 9 IEEE, 2021, pp. 23409–23423
- “Intriguing properties of neural networks”, 2014 arXiv:1312.6199 [cs.CV]
- Pin Wu, Yang Yang and Xiaoqiang Li “Image-into-image steganography using deep convolutional network” In Advances in Multimedia Information Processing–PCM 2018: 19th Pacific-Rim Conference on Multimedia, Hefei, China, September 21-22, 2018, Proceedings, Part II 19, 2018, pp. 792–802 Springer
- Han Xiao, Kashif Rasul and Roland Vollgraf “Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms” In CoRR abs/1708.07747, 2017 arXiv: http://arxiv.org/abs/1708.07747
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
Top Community Prompts
Collections
Sign up for free to add this paper to one or more collections.