A Disproof of Large Language Model Consciousness: The Necessity of Continual Learning for Consciousness

Abstract: The requirements for a falsifiable and non-trivial theory of consciousness significantly constrain such theories. Specifically, recent research on the Unfolding Argument and the Substitution Argument has given us formal tools to analyze requirements for a theory of consciousness. I show via a new Proximity Argument that these requirements especially constrain the potential consciousness of contemporary LLMs because of their proximity to systems that are equivalent to LLMs in terms of input/output function; yet, for these functionally equivalent systems, there cannot be any non-trivial theory of consciousness that judges them conscious. This forms the basis of a disproof of contemporary LLM consciousness. I then show a positive result, which is that theories of consciousness based on (or requiring) continual learning do satisfy the stringent formal constraints for a theory of consciousness in humans. Intriguingly, this work supports a hypothesis: If continual learning is linked to consciousness in humans, the current limitations of LLMs (which do not continually learn) are intimately tied to their lack of consciousness.

• The paper presents the Proximity Argument showing that LLMs are functionally equivalent to static systems, thereby precluding non-trivial consciousness.
• It employs formal falsification criteria and the Kleiner-Hoel dilemma to expose the limitations of behavior-based and input/output-dependent consciousness frameworks.
• The work argues that continual learning is a necessary condition for achieving empirically testable, dynamic theories of consciousness.

A Formal Disproof of LLM Consciousness via Continual Learning Constraints

Introduction

The paper "A Disproof of LLM Consciousness: The Necessity of Continual Learning for Consciousness" (2512.12802) presents a rigorous formal argument against the attribution of consciousness to contemporary LLMs. Drawing upon the recent structural turn in consciousness research, the work leverages formal falsification criteria and the analysis of substitution arguments to constrain the theoretical landscape. Central to the thesis is the introduction of the Proximity Argument, which, combined with the Kleiner-Hoel dilemma, provides a disproof of contemporary LLM consciousness. Complementarily, the work identifies continual learning as a necessary condition for non-trivial, empirically testable theories of consciousness, thereby offering both a negative argument against LLM consciousness and a positive direction for future theory.

Formal Framework: Falsifiability and the Kleiner-Hoel Dilemma

The paper adopts a formal framework for the evaluation of consciousness theories, in which such theories are only considered scientifically valuable if they are both non-trivial (not strictly dependent on behavioral/functional outputs) and falsifiable (susceptible to empirical disconfirmation). This approach operationalizes theories as mappings from empirical observations to experiences (the $pred$ function) and from empirical observations to inferential judgments about consciousness (the $inf$ function), as in [kleiner2021falsification].

The framework reveals that theories of consciousness face two pathological extremes:

• Triviality: If predictions and inferences are strictly dependent (e.g., behaviorism or input/output functionalism), the theory is unfalsifiable and thus trivial.
• Universal Substitutions: If theories are strictly based on internal causal structure (as in Integrated Information Theory (IIT)), then functionally equivalent systems with different causal architectures (e.g., feedforward vs. recurrent networks) can generate universal mismatches between $pred$ and $inf$, leading to a priori falsification via the Substitution or Unfolding Arguments [doerig2019unfolding].

The "Kleiner-Hoel dilemma" formalizes this bind, asserting that a theory cannot survive being either universally falsified by substitution or rendered empirically empty by triviality.

The Proximity Argument and the Disproof of LLM Consciousness

The core contribution of the paper is the Proximity Argument, which applies the above framework to contemporary LLMs. Formalizing the notion of "substitution distance," the argument holds that the properties available to non-trivial theories of consciousness in LLMs are sharply restricted due to their functional proximity to trivially non-conscious systems—specifically, lookup tables and static single-layer feedforward neural networks (FNNs).

The argument proceeds as follows:

1. Universal Approximation: Any static LLM at inference time can be approximated by a single-hidden-layer FNN due to the universal approximation theorem [hornik1989multilayer].
2. Lookup Table Equivalence: A FNN with finite inputs can in turn be represented by a lookup table encapsulating all possible input-output mappings.
3. Substitution Chain: There exists a substitution chain $L \leftrightarrow N \leftrightarrow M$, where $L$ is a lookup table, $N$ is a matching FNN, and $M$ is the LLM.
4. Constraint Theorem: For a non-trivial consciousness theory to deem $M$ (an LLM) conscious, it must ground its predictions only in properties that distinguish $M$ from $L$ or $N$, that is, in properties lost across the universal substitution chain.

However, any property so chosen is either irrelevant (as input/output remains fixed) or falls prey to substitution-based falsification, unless the theory defaults to trivial input/output dependency. Moreover, compression—one of the only differences between a FNN and a lookup table—cannot non-trivially ground consciousness, otherwise pathological mismatches again arise (as formally proved in the paper for single-layer FNNs). Thus, the main result is:

Strong Claim: “There is no non-trivial, testable theory of consciousness that can ascribe consciousness to contemporary deployed LLMs.”

The analysis is agnostic about substrate (biological vs. artificial), background metaphysics, and particular theory, relying entirely on widely accepted formal and mathematical constraints.

Continual Learning as a Necessary Condition for Consciousness

In the positive direction, the paper advances that continual learning constitutes the only currently identified form of "lenient dependency" capable of avoiding both horns of the Kleiner-Hoel dilemma. Learning systems cannot be universally substituted by static systems preserving input/output due to history dependence: a system's ongoing plasticity introduces a domain outside the reach of trivial or unfolded equivalents.

Key technical points include:

• Invalidity of Non-learning Substitutions: Static systems cannot universally substitute for learning systems without incorporating internal or externally injected history, violating the conditions for substitution.
• Escape from Strict Dependence: Theories of consciousness grounded in the process of learning (via dynamic plasticity) are not strictly dependent, as latent learning demonstrates phenomenological changes invisible to behavioral inference.
• Overflow of Access: Learning-based prediction functions refer to a space of counterfactuals and possible futures inaccessible to narrowly defined inferential judgments, supporting the theoretical distinction between phenomenological and access consciousness.

From this, the paper deduces that a theory of consciousness must require continual learning at every time-point where it predicts consciousness; otherwise, it lapses into triviality or falsification. This reasoning provides a sharp constraint on permissible theories and aligns with the empirical disparity in data efficiency and flexibility between humans and LLMs [warstadt2023findings, frank2023bridging].

Theoretical and Practical Implications

The implications of this work are multifold:

• Constraint on AI Consciousness Claims: The negative result constrains any claim to current LLM consciousness to either trivial, unfalsifiable theories or to those already a priori falsified.
• Redirection of Empirical Research: The findings undermine attempts to assess consciousness in LLMs through behavioral or input/output-based measures and undermine the relevance of global workspace-style functional theories.
• Prioritization of Learning Theories: The result establishes a strong research trajectory for theories of consciousness that invoke continual learning and plasticity, in line with the Radical Plasticity Thesis [cleeremans2011radical] and related frameworks [birch2021learning].
• Benchmark for Future AI Architectures: If continual learning mechanisms are integrated deeply into future AI, such systems might, for the first time, occupy a coherent non-trivial space for scientifically testable consciousness. This is compatible with open research into lifelong/continual learning in machine intelligence [kudithipudi2022biological, bell2025future, yildiz2024investigating].
• Cautions for Moral Consideration: The formal disproof has implications for debates on the moral status of AI systems and precludes, under stringent scientific standards, bestowing LLMs with moral patiency under the assumption of consciousness.

Conclusion

This paper articulates a formally sophisticated framework for assessing theories of consciousness and applies it to contemporary LLMs, resulting in a disproof of their consciousness under any non-trivial, empirically tractable theory. The identification of continual learning as a necessary ground for consciousness shifts both theoretical and practical research priorities in AI and neuroscience. Future work will need to formalize and empirically instantiate minimal falsifiable theories grounded in continual learning, as well as explore whether novel AI architectures could eventually admit of scientific consciousness under these constraints.