The World Is Bigger! A Computationally-Embedded Perspective on the Big World Hypothesis

Abstract: Continual learning is often motivated by the idea, known as the big world hypothesis, that "the world is bigger" than the agent. Recent problem formulations capture this idea by explicitly constraining an agent relative to the environment. These constraints lead to solutions in which the agent continually adapts to best use its limited capacity, rather than converging to a fixed solution. However, explicit constraints can be ad hoc, difficult to incorporate, and may limit the effectiveness of scaling up the agent's capacity. In this paper, we characterize a problem setting in which an agent, regardless of its capacity, is constrained by being embedded in the environment. In particular, we introduce a computationally-embedded perspective that represents an embedded agent as an automaton simulated within a universal (formal) computer. Such an automaton is always constrained; we prove that it is equivalent to an agent that interacts with a partially observable Markov decision process over a countably infinite state-space. We propose an objective for this setting, which we call interactivity, that measures an agent's ability to continually adapt its behaviour by learning new predictions. We then develop a model-based reinforcement learning algorithm for interactivity-seeking, and use it to construct a synthetic problem to evaluate continual learning capability. Our results show that deep nonlinear networks struggle to sustain interactivity, whereas deep linear networks sustain higher interactivity as capacity increases.

• The paper establishes that computationally-embedded agents are inherently limited, confirming the big world hypothesis with capacity-constrained interactivity.
• It introduces a novel formal framework using universal-local environments modeled by Turing-complete, local Markovian rules to simulate agent behavior.
• Empirical findings reveal that while deep linear networks maintain high interactivity, deep nonlinear networks falter, underscoring the need for continual adaptation.

A Computationally-Embedded Perspective on the Big World Hypothesis

Introduction and Motivation

This paper advances the formalization of continual learning within the context of the "big world hypothesis," which posits that any agent's representational and computational capacity is inherently dwarfed by the complexity and breadth of its environment. Traditional approaches in RL typically treat agent and environment as separable modules, often with explicit, fixed constraints imposed on the agent's storage, expressivity, or computational resources. However, these ad hoc constraints do not always capture the fundamental and persistent asymmetry between agent and environment. Instead, the authors propose a computationally-embedded perspective, modeling agents as automata inherently situated within, and thus simulated by, a computationally universal and local environment.

Formal Framework: Universal-Local Environments and Embedded Agents

The paper introduces the construct of the universal-local environment, encompassing two core properties:

• Universal computation: The environment is modeled as a Turing-complete, algorithmic Markov process over a countably infinite state space, ensuring the capability to simulate any computable agent or process.
• Uniform locality: State transitions are specified by local, homogeneous Markovian rules, facilitating the embedding of finite automata (agents) whose state is restricted to a finite, bounded region, akin to a cellular automaton such as Conway's Game of Life.

Within this environment, agents are formally embedded as local automata whose I/O interface forms a boundary via which all interaction with the rest of the environment occurs. This embedding implies that the agent's capacity is always strictly limited (finite internal state), even as the environment's state space remains unbounded.

A key formal result is that any such embedded automaton is constrained in the languages it can recognize and the sequences it can effect, with strict upper bounds on behavioral complexity imposed by its finite (potentially parameterized) state. This forms a direct, structural instantiation of the so-called "big world hypothesis": regardless of how the agent is scaled, the world (environment) "remains bigger."

Interactivity: A Computational Measure for Continual Adaptation

Recognizing the limitations of mutual information and Shannon entropy-based objectives in nonparametric, sequence-level agent modeling, the authors introduce interactivity—a capacity-relative, agent-centric measure inspired by Kolmogorov complexity. Specifically, interactivity is the difference between the unconditional and conditional algorithmic complexity (relative to the environment's reference machine) of the agent's future behavior sequence, conditioned on its past interaction history.

Formally, for a sequence of future interaction tuples (inputs, outputs) $b_{t:t+T-1}$ and history $b_{0:t-1}$:

$$\mathbb{I}_{T} = \mathbb{K}_{\mathcal{E}}(b_{t:t+T-1}) - \mathbb{K}_{\mathcal{E}}(b_{t:t+T-1} | b_{0:t-1}).$$

High interactivity therefore requires agents to produce complex (algorithmically incompressible) behavioral sequences that remain predictable from their past—a formalization that subsumes the plasticity-stability trade-off central to continual learning. Notably, the constructive results demonstrate that, for any given internal capacity, there exist upper bounds on the achievable interactivity, and that optimality in the big world setting obligates continual adaptation—agents that cease learning are provably suboptimal.

Model-Based RL Approach to Maximizing Interactivity

Since Kolmogorov complexity is uncomputable for arbitrary automata, practical instantiation of interactivity maximization necessitates approximate surrogates. The authors operationalize the agent's behavioral complexity as prediction error under a value function, leveraging a distortion-rate perspective. They instantiate a reinforcement learning algorithm that leverages model-based rollouts:

1. Predictive value function: The agent learns a value function via temporal-difference updates to forecast its own future I/O sequences.
2. Conditional vs. unconditional error: The agent computes static and dynamic prediction errors respective to its history and present policy parameters.
3. Meta-gradient optimization: A bi-level optimization process is employed wherein the policy is meta-updated to maximize the difference between static and dynamic error (interactivity), leveraging meta-gradients over the differentiable value function.

This approach self-generates non-stationarity, as the agent's policy and value function continue to evolve, sidestepping traditional stationarity assumptions or fixed environment task boundaries.

Empirical Findings

The empirical analysis focuses on a synthetic environment-free continual learning evaluation (self-prediction), isolating the core agent dynamics without additional confounds. Key results include:

• Deep nonlinear networks (ReLU, MLPs) fail to sustain high interactivity, exhibiting rapid collapse in the bi-level optimization process.
• Deep linear networks maintain high interactivity, with measurable gains achieved through capacity increases (width, depth). The action sequences generated by these models demonstrate structured, non-stationary (yet locally predictable) trajectories that maximize agent-relative interactivity.

These findings reinforce that interactivity maximization, unlike standard RL or supervised tasks, presents unique demands on representation and adaptation. Particularly, plasticity and stability are naturally entangled: only methods that can preserve the value function's capacity for continual rapid adaptation succeed.

Theoretical and Practical Implications

The framework introduced here has several significant theoretical consequences:

• Formalizes the continual learning desideratum: The agent is necessarily suboptimal when learning ceases, generalized across computational models.
• Eliminates reliance on ad hoc or externally imposed constraints: Agent limitations arise immediately from their embedding in a richer, unbounded environment.
• Invites novel intrinsic motivation frameworks: Interactivity generalizes and aligns with, but is distinct from, information-seeking and empowerment-based objectives (e.g., empowerment [klyubin2005empowerment], variational intrinsic control [gregor2016variational]), providing sequence-level, agent-relative measures robust in the presence of a 'big world'.

Practically, the environment-free evaluation task proposed opens the possibility for benchmark tasks that directly probe continual adaptation capacity absent predefined non-stationarities or explicit task boundaries—a distinctive approach compared to most current continual learning evaluations.

Speculations on Future Directions

This computational embedding perspective potentially generalizes to multi-agent scenarios, adaptive meta-learning, and lifelong learning paradigms where agent-environment distinctions are inherently porous. Extending the notion of interactivity as an auxiliary reward for exploration or as a constraint in RL holds promise, although estimation in high-dimensional, real-world environments will require scalable surrogates for algorithmic information.

Refining methods to scale meta-gradient optimization, stabilize nonlinear networks under continual interactivity maximization, and analyzing the interplay between environment structure, agent-based prediction, and adaptation algorithms constitute important open directions.

Conclusion

By casting continual learning and lifelong adaptation as emergent phenomena of computationally-embedded agents in universal-local environments, this work rigorously substantiates the big world hypothesis. The implicit, capacity-relative constraints of embeddedness and the introduction of interactivity as a computational objective yield new theoretical guarantees for continual adaptation and suggest practical tools for the development and evaluation of genuinely continual reinforcement learning algorithms. The empirical evidence underscores the nuanced demands of sustaining adaptation under such a regime, emphasizing the limitations of current deep nonlinear architectures compared to linear counterparts.

---

Reference: "The World Is Bigger! A Computationally-Embedded Perspective on the Big World Hypothesis" (2512.23419)