Seeing Faces in Things: A Model and Dataset for Pareidolia

Abstract: The human visual system is well-tuned to detect faces of all shapes and sizes. While this brings obvious survival advantages, such as a better chance of spotting unknown predators in the bush, it also leads to spurious face detections. Face pareidolia'' describes the perception of face-like structure among otherwise random stimuli: seeing faces in coffee stains or clouds in the sky. In this paper, we study face pareidolia from a computer vision perspective. We present an image dataset ofFaces in Things'', consisting of five thousand web images with human-annotated pareidolic faces. Using this dataset, we examine the extent to which a state-of-the-art human face detector exhibits pareidolia, and find a significant behavioral gap between humans and machines. We find that the evolutionary need for humans to detect animal faces, as well as human faces, may explain some of this gap. Finally, we propose a simple statistical model of pareidolia in images. Through studies on human subjects and our pareidolic face detectors we confirm a key prediction of our model regarding what image conditions are most likely to induce pareidolia. Dataset and Website: https://aka.ms/faces-in-things

• The paper introduces a novel dataset of 5,000 annotated pareidolic images, significantly advancing the study of non-traditional face detection.
• The paper applies empirical evaluations with the RetinaFace model to highlight a marked performance gap between human and machine detection of pareidolic faces.
• The paper proposes mathematical models that predict peak pareidolia at medium image complexity, offering theoretical insights to improve face detection systems.

A Model and Dataset for Pareidolia: Investigating Face Detection Mechanisms

Authors of the paper "Seeing Faces in Things: A Model and Dataset for Pareidolia" introduce a comprehensive study of face pareidolia—a psychological phenomenon where random stimuli are perceived as faces—from a computational perspective. This work includes the construction of a novel dataset, "Faces in Things," comprising five thousand annotated images of pareidolic faces and the investigation of face detection in both human and machine vision systems.

Dataset Construction and Annotation

The dataset "Faces in Things" is meticulously curated and annotated, offering five thousand images with pareidolic faces identified through human-annotated bounding boxes. Detailed attributes of these faces, such as perceived emotion, gender, intentionality, and difficulty to spot, are also included. This dataset serves as a fundamental resource for examining the capabilities and limitations of face detection models when confronted with non-human faces.

Empirical Evaluation of Face Detection Models

The authors employ the RetinaFace model—a state-of-the-art face detection algorithm trained on the WIDER FACE dataset—to determine its performance in detecting pareidolic faces. Initial results reveal a significant gap in the performance of these models compared to human perception, indicating that the current face detection models, though highly effective in recognizing human faces, struggle with detecting pareidolic faces. Fine-tuning these models on the pareidolic face dataset significantly improves their detection performance, but still does not completely bridge the gap to human performance.

Exploring Mechanisms Underlying Pareidolia

To further explore why current face detection models fail to exhibit pareidolia as humans do, the study investigates various training data interventions. Incorporating animal faces into the training data markedly enhances the performance of face detection models in recognizing pareidolic faces, suggesting that evolutionary factors requiring the detection of diverse faces in natural environments might play a role in the phenomenon of pareidolia. This is corroborated by the observation that Rhesus monkeys also display pareidolic behavior, pointing towards a broader biological basis for the effect.

Mathematical Models of Pareidolia

Beyond empirical studies, the paper proposes two mathematical models to describe the phenomenon of pareidolia: a Gaussian model and a higher-level feature model. Both models predict a peak in pareidolic detection at a medium level of image complexity. Empirical psychophysics experiments conducted with human subjects confirm this prediction, as do evaluations with machine models fine-tuned on the pareidolic face dataset. These models offer a theoretical framework that can guide future explorations into the mechanisms of face detection and pareidolia.

Practical and Theoretical Implications

The findings of this study have several practical and theoretical implications. Understanding the conditions that give rise to pareidolia can aid in designing more robust face detection systems that minimize false positives in security applications. Additionally, insights from this research can contribute to the creation of better human-computer interaction systems where machines can understand and replicate human perceptual phenomena more accurately. Furthermore, the dataset and models provided can serve as foundational tools for future research in both computer vision and cognitive psychology.

Conclusion

The paper provides substantial contributions to the study of face pareidolia by presenting a well-annotated dataset and proposing both empirical and theoretical models to understand the phenomenon. It highlights crucial differences between human and machine face detection capabilities and suggests evolutionary and psychophysical reasons behind these differences. The research opens new avenues for exploring how face detection can be improved and how machine vision systems can be made more human-like in their perception abilities. Future work in AI can leverage these insights to develop systems that are not only more accurate but also more aligned with human perceptual experiences.