Normality and the Turing Test

Abstract: This paper proposes to revisit the Turing test through the concept of normality. Its core argument is that the statistical interpretation of the normal--understood as the average both in the normative and mathematical sense of the term--proves useful for understanding the Turing test in at least two ways. First, in the sense that the Turing test targets normal/average rather than exceptional human intelligence, so that successfully passing the test requires building machines that "make mistakes" and display imperfect behavior just like normal/average humans. Second, in the sense that the Turing test is a statistical test where judgments of intelligence are never carried out by a single "average" judge (understood as non-expert) but always by a full jury. As such, the notion of "average human interrogator" that Turing talks about in his original paper should be understood primarily as referring to a mathematical abstraction made of the normalized aggregate of individual judgments of multiple judges. In short, this paper argues that the Turing test is a test of normal intelligence as assessed by a normal judge characterizing the average judgment of a pool of human interrogators. Its conclusions are twofold. First, it argues that LLMs such as ChatGPT are unlikely to pass the Turing test as those models precisely target exceptional rather than normal/average human intelligence. As such, they constitute models of what it proposes to call artificial smartness rather than artificial intelligence per se. Second, it argues that the core question of whether the Turing test can contribute anything to the understanding of human cognition is that of whether the human mind is really reducible to the normal/average mind--a question which largely extends beyond the Turing test itself and questions the conceptual underpinnings of the normalist paradigm it belongs to.

• The paper reinterprets the Turing test by emphasizing that AI should replicate typical human imperfections rather than exceptional intelligence.
• It employs a statistical framework inspired by Quetelet’s 'Average Man' theory to assess machine indistinguishability through collective human judgment.
• The study suggests that aligning AI responses with average human behavior may enhance the realism and usability of machine-human interactions.

Normality and the Turing Test: An Expert Overview

Reinterpreting the Turing Test

The paper "Normality and the Turing Test" provides a sophisticated reevaluation of the Turing test, embedding it within the concept of normality in both a normative and statistical sense. Turing's original test aimed to measure a machine's ability to exhibit behavior indistinguishable from a human's through text-based interaction. However, this paper argues that Turing did not focus on exceptional human intelligence but instead targeted normal or average human intelligence. This implies machines aiming to pass the Turing test should replicate typical human imperfections to embody realistic human behavior.

Normality as a Statistical Framework

The paper asserts that the Turing test is inherently statistical. Its judgment relies on aggregating responses from multiple interrogators, not just a single judge, to mitigate individual biases or errors in judgment. This aligns with the historical application of statistics in evaluating human characteristics, similar to Quetelet's "Average Man" theory. In a statistical context, achieving 50% indistinguishability implies a machine's responses should mathematically align with the averaged, typical human behavior as judged by a pool of varied human interrogators.

Revisiting LLMs

The paper argues that models like ChatGPT, despite their advanced capabilities, target exceptional rather than average intelligence. These models are tailored to perform with high efficiency and correctness, deviating from the imperfect nature of average human intelligence expected in the Turing test. Thus, despite their proficiency, such models are said to be unlikely to pass the Turing test as they embody artificial smartness rather than intelligence in Turing's original context.

Implications for Artificial Intelligence

This reconsideration of the Turing test suggests a nuanced understanding of AI's goalposts. Instead of striving for exceptional intelligence or perfection, AI models might better serve if aligned more closely with average human behavior when the goal is to simulate human-like responses effectively. This orientation, towards the normal rather than the exceptional, might redefine success within AI systems designed to interact indistinguishably with humans.

Limitations and Perspectives

The concept of normality, while providing significant insight into the Turing test, also poses challenges due to its reliance on cultural and historical averages, which may not universally apply across different populations. Thus, while the statistical interpretation validates the test's robustness, it also highlights its limitations in capturing the full diversity of human intelligence. Future AI development might need to consider these cultural variations to achieve genuine indistinguishability aligned with Turing's vision.

Conclusion

Overall, "Normality and the Turing Test" offers a compelling reinterpretation of the Turing test, suggesting its true focus is on modeling average human behavior rather than its exceptional counterparts. By framing the test within a broader statistical and normative context, the paper challenges AI researchers to reconsider how intelligence is modeled and assessed in artificial systems. This shift may pave the way for more nuanced and contextually relevant applications of artificial intelligence, focusing on usability and human-like interaction over mere computational prowess.