Resource-Rational Mechanism Selection

• Resource-rational mechanism selection is a decision-theoretic framework that selects among cognitive and algorithmic processes by trading off performance and complexity.
• It formalizes complexity costs using Lagrangian objectives and KL divergence, guiding mechanism choice in reinforcement learning and perceptual decision-making.
• The framework drives algorithmic meta-learning and explains behavioral phase transitions, with empirical evidence from neuropsychological and developmental studies.

Resource-rational mechanism selection is a formal decision-theoretic framework for selecting among candidate cognitive or algorithmic mechanisms by trading off task performance against the computational or representational complexity required to implement those mechanisms. The core hypothesis is that, under real-world constraints on computational, storage, or descriptive resources, agents (biological or artificial) maximize expected effectiveness subject to costs that are typically proportional to complexity. This framework has been applied to domains including reinforcement learning and perceptual decision-making, yielding precise, testable predictions about both algorithmic design and observed human and animal behavior (Binz et al., 2022, Lee et al., 30 Sep 2025).

1. Formal Resource-Rational Objectives

At the heart of resource-rational mechanism selection is an explicit objective function that penalizes complexity. In reinforcement learning, the problem is often posed by meta-learning over a class of algorithms π, each described by parameters W. The Lagrangian objective is

$$L(\pi) = -\mathbb{E}_{\tau \sim \pi}[R(\tau)] + \lambda \cdot \mathrm{DL}(\pi),$$

where $R(\tau)$ is the return of trajectory $\tau$, $\mathrm{DL}(\pi)$ is the description length (in bits or nats) of the policy π, and λ determines the cost-weighting. When π is implemented by a distribution over parameters $q(W|\Lambda)$ with prior $p(W)$, $\mathrm{DL}$ is computed as a KL divergence: $$\mathrm{DL}_{\text{nats}}(\pi) = \operatorname{KL}[q(W|\Lambda) \Vert p(W)].$$ This resource penalty can be re-expressed as a constrained maximization or a dual objective involving Lagrange multipliers. Analogous principles govern perceptual mechanism selection, where the agent chooses mechanism $M$ to maximize

$$M^* = \arg\max_M \Bigl\{ \mathbb{E}_{p(x,y)} [\mathbb{E}_{d \sim f_M(x)} [U(d, y)]] - \lambda C(M) \Bigr\},$$

with $C(M)$ the complexity cost (e.g., the number of stored scalars) and $U(d, y)$ the utility of decision $d$ if the correct response is $y$ (Lee et al., 30 Sep 2025).

2. Complexity Measures and Encoding Schemes

Resource cost is typically operationalized as either algorithmic or representational. In algorithmic domains, e.g., deep RL, description length is measured in bits required to encode parameter vector $W$ under a coding prior, as given by

$$\mathrm{DL}_{\text{bits}}(\pi) = \mathrm{KL}[q(W|\Lambda) \Vert p(W)] / \ln 2.$$

Variational distributions (e.g., independent Gaussians) are commonly used for $q(W|\Lambda)$, with expected code-length estimated analytically or by Monte Carlo (Binz et al., 2022). In perceptual decision-making, complexity $C(M)$ grows linearly with the number of stored evidence values (e.g., $C(\mathrm{summary})=1$, $C(\mathrm{population})=4$ for four options) (Lee et al., 30 Sep 2025).

3. Algorithmic Meta-Learning and Mechanism Discovery

The mechanism-selection problem is solved by meta-learning both the policy and its encoding under resource constraints. In RL, this is achieved by training an exploration algorithm (e.g., RR-RL²) via on-policy actor-critic, augmented with dual-gradient optimization for Lagrange multipliers enforcing a code-length budget:

Initialize Λ, β for meta-iteration = 1…N do sample task ω ~ p(ω) sample W ~ q(W|Λ) run π_W for H steps, collect (a_t,r_t) compute reward-advantage A_t accumulate grad_Λ[-E_q[R] + β(KL−C)] accumulate grad_β[β(KL–C)] update Λ, β accordingly end for

The resulting procedure discovers which (encoded) exploration strategy maximizes reward under limited bits—automatically recovering a spectrum from cheap heuristics to complex near-optimal mechanisms (Binz et al., 2022).

In perception, this process is modeled by comparing log-likelihoods and AICs across candidate mechanisms of varying complexity as task demands escalate, with the penalty parameter λ tuned across experimental conditions (Lee et al., 30 Sep 2025).

4. Mechanistic Transitions and Resource-Driven Behavioral Phases

Resource constraints induce distinct algorithmic and behavioral phases. In RL, with small budgets ($C < 100$ nats), agents recover Boltzmann-style value-based random exploration; for intermediate budgets ($100 \leq C \leq 1000$ nats), behavior shifts towards Thompson sampling; at high budgets ($C > 1000$ nats), UCB-style directed exploration emerges. This mechanism selection is captured by fitting meta-learned data to a hybrid probit regression, with phase transitions reflected in the weights $w_1$, $w_2$, $w_3$ assigned to value, uncertainty, and gain terms (Binz et al., 2022).

In perceptual tasks, mechanism transitions are observed as tasks progressively require more complex strategies to maximize accuracy:

• In low-demand phases, simple mechanisms (e.g., summary model, $C=1$) are favored.
• With tasks designed to defeat shortcuts, more complex representations (e.g., population model, $C=4$) become optimal (Lee et al., 30 Sep 2025).

Empirical model comparison (e.g., via AIC) across experiments confirms that human subjects adapt mechanism complexity in line with the resource-rational criterion.

Budget/Constraint	RL Mechanism (RR-RL²)	Perceptual Mechanism
Small ($C \ll 100$)	Boltzmann/random	Summary encoding (max only)
Intermediate ($C \sim 500$)	Thompson sampling	Two-highest encoding
Large ($C > 1000$)	UCB/directed exploration	Full population encoding

5. Empirical Applications and Behavioral Parallels

Resource-rational mechanism selection accounts for a range of neuropsychological and developmental phenomena. In the Iowa Gambling Task, low-budget models ($C \approx 100$ nats) mimic vmPFC-lesioned behavior, over-weighting high-variance decks and perseverating on risky options (~70% high-risk choices), while large budgets ($C \approx 10,000$ nats) reproduce healthy performance (~20% high-risk choices) (Binz et al., 2022).

In developmental paradigms such as the Horizon Task, increasing budget (mimicking cognitive maturation) enhances directed, strategic exploration without affecting random exploration—the pattern observed in adolescent data ($F_{1,194} \approx 17.5$, $p < 0.001$ for directed, constant for random) (Binz et al., 2022).

In perceptual experiments, increasing task complexity and manipulating reward structures induce shifts in mechanism selection from summary to two-highest to full population—precisely as predicted by the linear resource-penalty framework. These results demonstrate that apparent suboptimality in human perceptual inference often reflects rational adaptation to resource constraints rather than inherent cognitive limits (Lee et al., 30 Sep 2025).

6. Generalization and Theoretical Implications

The framework of resource-rational mechanism selection generalizes across domains, provided one can specify a family of candidate tasks, a parametric form for representational or algorithmic mechanisms, and an information-theoretic complexity penalty. Applications span contextual bandits, grid-worlds, structured control, multi-agent systems, and perceptual decision-making.

This approach prescribes:

1. Specification of a flexible policy or representational scheme,
2. Imposition of an explicit complexity or code-length penalty (e.g., KL-divergence, scalar count),
3. Meta-learning or empirical comparison across mechanisms and code costs,
4. Interpretation of parameter $\lambda$ (or bit budget $C$) as indexing a continuum of resource allocation regimes,
5. Quantitative alignment of mechanism phase transitions with empirical behavior across neuropsychological, developmental, and cultural variations (Binz et al., 2022, Lee et al., 30 Sep 2025).

A key implication is that experimental and cognitive modeling interpretations must account for the possibility of resource-rational selection among available mechanisms, rather than inferring cognitive limitations from apparent suboptimality. Thus, both experimental design and model selection procedures should explicitly test whether low-complexity strategies suffice to explain performance before attributing failures to cognitive or neural constraints.

7. Summary and Core Equations

Resource-rational mechanism selection offers a coherent, empirically validated account of how agents adaptively balance task performance and resource constraints:

• Resource-rational criterion:

$$M^* = \arg\max_M\Bigl\{\mathbb{E}_{p(x,y)}[\mathbb{E}_{d\sim f_M(x)}[U(d,y)]] - \lambda\,C(M)\Bigr\}$$

• Dual-constrained meta-objective (for RL):

$$\max_{\Lambda} \; \mathbb{E}_{q(W|\Lambda)p(\omega)\pi(\cdot|\cdot,W)}[R(\tau)] \quad \text{subject to} \quad \mathrm{KL}[q(W|\Lambda) \Vert p(W)] \leq C$$

• Posterior over mechanisms (perception):

$$P(M \mid D) \propto P(D\mid M) P(M) \exp(-\lambda C(M))$$

• Likelihood-based model selection:

$\mathrm{AIC} = 2k - 2\log \mathcal{L}$

This framework is central to understanding the computational rationality underlying both artificial and natural agents, solidifying resource constraints as a foundational principle in the selection of cognitive and algorithmic mechanisms (Binz et al., 2022, Lee et al., 30 Sep 2025).