Do Zombies Understand? A Choose-Your-Own-Adventure Exploration of Machine Cognition

Abstract: Recent advances in LLMs have sparked a debate on whether they understand text. In this position paper, we argue that opponents in this debate hold different definitions for understanding, and particularly differ in their view on the role of consciousness. To substantiate this claim, we propose a thought experiment involving an open-source chatbot $Z$ which excels on every possible benchmark, seemingly without subjective experience. We ask whether $Z$ is capable of understanding, and show that different schools of thought within seminal AI research seem to answer this question differently, uncovering their terminological disagreement. Moving forward, we propose two distinct working definitions for understanding which explicitly acknowledge the question of consciousness, and draw connections with a rich literature in philosophy, psychology and neuroscience.

• The paper introduces a thought experiment with the chatbot ZZZ, challenging the idea that consciousness is required for functional understanding.
• It employs the philosophical zombie analogy to contrast observable performance in NLP with the presence of genuine conscious experience.
• The paper outlines future research directions, suggesting that insights from neuroscience may further illuminate AI cognition debates.

Do Zombies Understand? Exploring Machine Cognition Through a Philosophical Lens

Introduction to the Thought Experiment

Recent advancements in LLMs have reignited discussions around machine understanding and cognition. A central debate is whether these models can be said to truly "understand" in any meaningful sense, a question that inevitably leads to considerations of consciousness and subjective experience. In exploring this debate, the paper introduces a thought experiment centered around a hypothetical chatbot, termed Z𝑍Z, which demonstrates unprecedented success across all possible NLP benchmarks, despite ostensibly lacking any form of consciousness. The experiment serves to probe the role of consciousness in machine understanding, drawing on differing perspectives within AI research and adjacent fields.

Background on Philosophical Zombies

Philosophical zombies, a concept derived from metaphysical discussions, are beings indistinguishable from humans in their behavior but devoid of conscious experience. Applied to AI, the paper posits an equivalent entity, Z𝑍Z, that mirrors human linguistic performance without the subjective experience, framing a discourse on whether such an entity genuinely understands. This analogy leverages historical philosophical debates, notably Chalmers’ work on consciousness, to scrutinize AI’s potential for understanding.

Understanding Without Consciousness

The paper delineates a functional definition of understanding, reliant solely on observable performance rather than subjective experience. Under this paradigm, a model like Z𝑍Z achieves understanding by performing tasks to a human or superior level, detached from any consciousness. Historical benchmarks of AI prowess, such as chess and Go, exemplify the shift in goalposts that accompanies technological progress, making Z𝑍Z an ultimate benchmark for functional understanding. This perspective views understanding as attainable through incremental progress on NLP tasks, harboring Z𝑍Z as the epitome of a chatbot that functionally understands every aspect of human language without consciousness.

The Necessity of Conscious Experience

In contrast, another school of thought insists on consciousness as indispensable for genuine understanding. This perspective, deeply rooted in philosophical and cognitive science traditions, demands both functional competence and subjective experience. The paper references key discussions, including Turing’s contemplations and Searle’s Chinese Room argument, to emphasize that AI, irrespective of its functional successes, lacks understanding if devoid of consciousness. This viewpoint advocates for a conscious understanding definition and suggests that future AI research might glean insights from neuroscience, particularly theories regarding the neural correlates of consciousness.

Addressing Alternative Perspectives

The paper acknowledges and responds to potential objections to its premise, including the relevance of Z𝑍Z’s implementation details and the conceivability of Z𝑍Z itself. It debates these points within the scopes of both functional and conscious understanding, ultimately reinforcing the bifurcation of these viewpoints while inviting further discourse on what constitutes understanding in AI.

Conclusive Thoughts and Future Directions

Concluding, the paper does not seek to arbitrate between these divergent perspectives but rather to explicate the underlying assumptions about consciousness and understanding in current AI debates. By delineating two distinct definitions of understanding, it opens avenues for more nuanced research agendas and theoretical discussions. Additionally, the paper suggests that this framework can be extended to other aspects of cognition in AI, like empathy, further exploring the significance of consciousness in attributing human-like traits to machines.

Limitations and Invitations for Future Discussion

The paper’s approach, grounded in philosophical analysis and thought experiments, inherently faces limitations in empirically addressing the complexities of consciousness and machine understanding. It acknowledges potential overlooked viewpoints and encourages a broader engagement with these foundational questions, signaling an ongoing dialogue within the AI research community on the nature of machine cognition and the elusive role of consciousness therein.