Enumerability and Learning Limitations

• Enumerability is defined as algorithmically listable structures, which form the basis for representing hypotheses in computability and learning theory.
• Studies reveal that key learning models, from Gold-style inductive inference to neural networks, face inherent limits due to their reliance on enumerable representations.
• Practical implications include challenges in designing efficient PAC learners and neural systems, prompting the exploration of hybrid symbolic-neural approaches.

Enumerability and Learning Limitations

Enumerability captures the property of being generated, listed, or described by an effective algorithm, and is central to both classical computability theory and learning-theoretic frameworks. In the context of learning, enumerability underpins what can be learned, the form that hypotheses or concept classes must take, and the barriers—both combinatorial and algorithmic—that delimit learnability. Across recursion theory, inductive inference, statistical learning, and neural networks, the limitation that only enumerable (recursively enumerable, computably enumerable, or recursively enumerably representable) structures can be manipulated or distinguished by effective agents leads to deep, unavoidable constraints on identification, approximation, and generalization.

1. Enumerability in Computability and Learning

Enumerability is formalized via recursively enumerable (r.e.) sets: a set $S \subseteq \mathbb{N}$ is r.e. if there exists an effective, partial algorithm (Turing machine) that lists its elements. Decidable (recursive) sets are a strict subclass, where membership is algorithmically testable. Computable functions correspond to total output by algorithms; partial computable (recursive) functions relax totality, permitting divergence (Prost, 2019). All transmissible, algorithmically processable knowledge must be encoded as subsets or functions on such an enumerable domain.

Gold's paradigm of identification in the limit utilizes enumerability by modeling the learner's hypotheses as indices to r.e. objects, and the positive data stream as a computable text. The learning machine's task is to stabilize in its guesses—after finitely many mind changes—to an index of the target set or structure (Prost, 2019, Mude, 2013). More generally, the effectiveness of learning methods, such as query learning or computational experimentation, are bounded by the logical complexity (r.e., co-r.e., etc.) of representing and distinguishing hypotheses (Prost, 2019).

2. Foundations: Limit Computability and Inductive Inference

The notion of "Computable in the Limit" (Mude, 2013) extends classical recursion theory. A function or property $P_p(s)$ is computable in the limit if there exists a partial recursive function $f_p(s, t)$ such that $f_p(s, t)$ may change its output finitely many times before stabilizing on the correct answer for each $s$:

$$P_p(s) = x \iff \exists u\ [f_p(s, u) = x \wedge \forall t \geq u\ (f_p(s, t) \downarrow \implies f_p(s, t) = x)]$$

This formalizes a type of learning where guesses may be revised until a correct answer is reached, but no further corrections are needed—a direct expression of "learning in the limit" (Mude, 2013). The Normal Form Theorem for such functions introduces a "last" search operator, a λ instead of Kleene's μ, picking the greates y where the function's value changes, requiring at most finitely many mind changes.

These concepts allow enumeration of complex objects—such as minimal Kolmogorov descriptions, incompressible numbers, or divergence points of partial recursive functions—via dovetailed, stage-wise refinements, yet retain inherent non-computability for certain tasks. Critically, no limit-computable enumeration can yield a complete list of indices for all Turing machines that compute a given total function, by a diagonalization argument (Mude, 2013). The interplay between effective enumeration and the stability of learning is thus intimately linked to the limits of recursion.

3. Gold-Style Language and Structure Learning: Barriers and Characterizations

In Gold-style inductive inference, the learnability of language families is strictly delimited by their enumerability and structural properties. If a family contains all finite languages and at least one infinite language, no computable learner given only positive data can identify every target in the limit—this is Gold's classical negative result (Prost, 2019, Alves, 2021).

Extending this, Angluin provided a combinatorial characterization: a family is learnable in the limit if and only if no infinite strictly increasing chain of languages within the family converges arbitrarily closely (in a chosen metric) to a target language. The existence of "locking data sets"—finite sets whose appearance fixes the learner's hypothesis—becomes both necessary and, in families containing all finite languages, sufficient for learnability (Alves, 2021). This formalizes the intuition that only when finite, enumerable evidence is sufficient to fix the target can enumeration-based identification succeed.

When learning computable structures such as equivalence relations, analogous characterizations hold: finite separability (the capacity to distinguish isomorphism types by finite data) precisely describes InfEx-learnability (learning up to isomorphism from an informant) for families without infinite classes (Fokina et al., 2019). Enumerability of candidates is necessary, but not sufficient without further separability.

Limitations sharpen when learning under additional presentation constraints, such as r.e. equivalence relations (η), yielding subtle hierarchies of learnability—behaviourally correct, vacillatory, explanatory, confident—with separations determined by the structure of the enumeration imposed by η (Belanger et al., 2020).

4. Enumerability in Algorithmic Learning Theory and Arithmetic Complexity

Characterizations of learnability for uniformly computably enumerable (u.c.e.) families can be precisely located in the arithmetic hierarchy:

• Finite learning: $\Sigma_2^0$-complete
• Learning in the limit: $\Sigma_4^0$-complete
• Behaviourally correct or anomalous in the limit: $\Sigma_5^0$-complete

Failure of learning is always witnessed by a $\Delta_2^0$ (limit-computable) enumeration on which the learner fails, highlighting that nonlearnability in this setting cannot be "hidden" higher in the hierarchy (Beros, 2013). The complexity of deciding learnability rises with the permissiveness of the convergence criterion (more errors allowed in the hypothesis stream). In all cases, effective enumeration both restricts the learner's access to hypotheses and forms the lower bound for the complexity of the identification meta-problem.

5. Enumerability Constraints in Practical and Hybrid Learning

When learning functions or languages with resource constraints (e.g., polynomial time), enumerability remains the central bottleneck for efficient learning. In teacher/learner/oracle models, the availability of an external "teacher" who can deliver a characteristic sample (a finite, identifying set) enables efficient identification within polynomial resources, whereas membership oracle access alone is insufficient for some highly enumerative families (Beros et al., 2015). These separations, witnessed by explicit family constructions, show that enumeration is not a mere formality, but profoundly affects feasible learning power.

In computable variants of PAC learning, recursively enumerable representability (RER) is necessary for effective hypothesis class specification. The effective VC-dimension (eVC) replaces the classical VC-dimension: uniform computable learnability (CPAC) is possible only when a computable witness exists for shattering finite sets. In general, eVC can be strictly greater than VC, breaking the equivalence between combinatorial complexity and learnability (Kattermann et al., 4 Nov 2025). Nonuniform learnability is restored for RER classes by allowing instance-dependent sample sizes (computable structural risk minimization), but any hope of efficient, uniform PAC learning collapses without enumerability or computable ERM.

6. Sequential Enumeration in Neural Networks: The Neural–Symbolic Gap

Modern neural network architectures, including LLMs, illustrate ongoing enumerability-related limitations in algorithmic learning. Sequential enumeration—exact counting or systematic production of repeated items—can be executed without error by symbolic (enumerable, serial-computation-based) systems, but presents a persistent challenge for standard neural models, especially those relying on attention and feed-forward architectures.

Empirical investigations reveal that, while scaling (increasing model size) leads to improved but gradual and approximate enumeration capability, no observed LLM spontaneously employs precise, serial counting procedures absent explicit stepwise prompting (Hou et al., 4 Dec 2025). Even with chain-of-thought induction, only explicit strategies achieve near-perfect accuracy (e.g., GPT-5, Gemini under explicit counting), while spontaneous performance remains far below. Embedding analysis uncovers analog-residue numerosity encoding (principal components sweeping out a smooth curve in token generation count) but no robust accumulator or discrete-state successor function.

Failures—early termination, over-counting, chunking heuristics, and copying mistakes—reflect the absence of symbolic compositional generalization. This strongly implicates the distributed, parallel, and non-enumerable inductive bias of neural architectures as a core limitation, with enumeration errors scaling as a power-law rather than showing abrupt phase transitions. Closing this gap requires hybrid symbolic–neural forms, external counter modules, or explicit architectural provisions for registering and terminating enumeration in a way compatible with Turing machine enumeration (Hou et al., 4 Dec 2025).

7. Implications and Open Directions

Enumerability entails fundamental, irreducible limitations in all effective learning scenarios. Identification in the limit, computable PAC learning, and neural sequence modeling each face distinct but structurally analogous barriers—rooted in the inability to algorithmically distinguish all infinite hypotheses absent finite, enumerable witnesses.

Characteristic or locking sets must be algorithmically enumerable for uniform identification. When such finite “cores” do not exist or cannot be synthesized efficiently, either learning collapses (diagonalization and nonlearnability) or exponential resource demands ensue (intractable characteristic sets or super-polynomial sample requirements) (Papazov et al., 18 Jun 2025).

Depending on the domain, workarounds include hybridizing inference protocols (teacher filters, time-bound trajectories), restricting to subclasses with computable separation (RER classes of finite eVC), or enriching observation (TBO, PTO signals for algorithmic learning of agents). However, in all cases, the boundary is determined by the enumerability properties of the hypothesis class, data, and the structure of the learning protocol.

Enumerability, therefore, remains the core conceptual and technical axis along which the possibility and efficiency of learning must be measured, and major future advances in both symbolic and subsymbolic learning hinge on new ways of embedding, leveraging, or bypassing its limitations.