A Theory of Appropriateness That Accounts for Norms of Rationality

Abstract: We propose a society-first theory of normative appropriateness where individuals, modeled as pre-trained actors with cognitive architectures analogous to LLMs, generate behavior via predictive pattern completion. Our theory posits that individuals act by completing distributed symbolic patterns based on context, answering questions such as "What does a person such as I do in a situation such as this?". This sense-making mechanism provides a parsimonious account of the key features of human norms: their context-dependence, arbitrariness, automaticity, dynamism, and their support from social sanctioning. It challenges rational-choice theories of social norms by accounting for their key features without needing to exogenously posit scalar rewards or preference relations. By distinguishing between explicit norms, which we associate with in-context adaptation, and implicit norms, which we associate with long-term memory, the theory reconceptualizes several foundational ideas in cognitive science. In particular, it gives an alternative account to the data traditionally seen as supporting dual-process models, and it flips the role of rationality, allowing us to construe it as adherence to culturally-contingent justification standards.

• The paper introduces a society-first computational model that employs LLM-inspired pattern completion to simulate norm formation and sanctioning.
• It rigorously formalizes normative behavior as emerging from culturally encoded predictive processes, challenging traditional rational-choice theory.
• The study highlights implications for AI alignment and social simulation by modeling how cultural priors and sanctioning dynamics shape decision-making.

A Society-First Theory of Appropriateness and Norms of Rationality

Introduction

The paper "A Theory of Appropriateness That Accounts for Norms of Rationality" (2603.14050) articulates a formal, computationally explicit 'society-first' theory of social order and normativity. Its core departure from methodological individualism is the modeling of individuals as culturally pre-trained actors, whose decentralized cognition—framed as predictive pattern completion via LLM-like architectures—internalizes and manifests social norms through situated sense-making. This approach rigorously recasts norms, rationality, and social order, challenging the exogenous parameterization of preferences and rational agency that underpins rational-choice explanations.

Core Mechanism: Predictive Pattern Completion via LLM Architectures

At the heart of the theory is the hypothesis that individual human decision-making, both in the mundane and the normative, is fundamentally a process of predictive pattern completion. This process is instantiated using a global workspace formalism: at each time $t$, the actor's cognition comprises a transient global workspace $\mathbf{z}_t$, in which contextually salient observations, memory retrievals, and internal queries are encoded as symbol sequences or 'assemblies.' An LLM-like pattern completion network $p$ autoregressively extends $\mathbf{z}_t$ by applying sequential or parallel 'summary functions'—prompting the actor to answer self-directed questions such as "What kind of situation is this?" and "What would a person like me do in a situation like this?" The final action $a_t$ is generated via a dedicated policy summary function, possibly conditioned on retrieved explicit memories. This entire cognitive cycle is both flexible and highly sensitive to cultural priors embedded in both long-term (model weights) and short-term (episodic memory, in-context learning) representations.

Figure 2: Schematic of the global workspace architecture, with parallel summary functions integrating perceptual, mnemonic, and premotor information, inspired by the neurocognitive global workspace model.

The architecture posits explicit modules mapping onto cortical, hippocampal, and pre-motor processes, supporting the claim that context-dependent, identity-conditioned sense-making mechanistically underlies norm adherence.

Reconceptualizing Norms, Rationality, and Sanctioning

Definition and Formalization of Norms

Normative behavior, in this theory, is characterized not by utility maximization but by contextually conventional patterns of social sanctioning—i.e., established mappings of approval and disapproval that operate via both direct social signals and third-party observation. This approach decouples normativity from scalar rewards, modeling sanctions as symbolic information that conditions future pattern completion in all (direct and indirect) observers, rather than as exogenous utilities.

Conventions emerge and stabilize through precedent-weighted reproduction in the population: actors are 'convention sensitive' if their response probabilities are counterfactually altered by widespread memory editing (e.g., replacing all observed instances of $a$ with $a'$ in historical context), with norm stability and adoption arising even in the total absence of explicit incentive specification. Crucially, arbitrariness in the content of norms is explained by symmetry-breaking in the formation of conventions, rather than by coordination equilibria with explicitly parameterized payoffs.

Sanctioning and Social Order

Sanctions in this model are formalized as contextual information that inversely modulates the probability of norm-violating completions. The theory accommodates not only direct, material sanctions but symbolic, communicative, and reputational ones—crucial for explaining third-party norm enforcement, rapid cultural drift, and the persistence of non-utilitarian or socially costly practices.

Implicit vs Explicit Norms and Learning

The architecture distinguishes between implicit norms (consolidated into model weights via repeated exposure) and explicit norms (retrieved ad hoc from episodic memory or in-context learning 'programming'). Explicit norms correspond to rules, laws, and articulated standards; implicit norms drive automaticity, efficiency, and the habitual nature of norm-conforming behavior. This distinction provides an explanatory framework for a broad array of dual-process phenomena observed in judgment, where automaticity is predicted to correlate with pattern consolidation and contextual predictability, while deliberative reasoning corresponds to multi-step sequence generation with memory-dependent chaining.

Explanatory Power: The Five Stylized Facts of Normative Cognition

The theory is developed to account for five empirically robust features of human norm-related behavior:

1. Context-dependence: Appropriateness is a function of situational cues, roles, and shifting identities—modeled as dynamic activation within the global workspace.
2. Arbitrariness: Norms are often arbitrary in content, behaving as conventionalized solutions reproducible under alternative historical path dependencies.
3. Automaticity: The majority of norm-conforming actions are guided by high-probability, short chain-of-thought pattern completions, explaining the robustness of social order under cognitive load and in familiar contexts.
4. Dynamism: Norm change is framed as a function of collective memory editing and cascades following positive feedback in sanctioning behavior, providing mechanistic accounts for both gradual evolution and rapid tipping-point dynamics.
5. Sanctioning: Social sanctioning is critical in establishing, maintaining, and shifting norms, not simply as reward modulation but as distributed information flow modulating population-level pattern completion tendencies.

Rationality Reframed as a Normative Technology

A central, bold claim is that rationality itself—understood as the application of explicit, consequence-evaluating, maximization procedures—is not foundational but emerges as a culturally contingent, explicit decision logic consolidated within certain epistemic communities (e.g., economics, engineering). The canonical sequence of rational choice (enumerate options, project consequences, select maximal expected value) is represented as just one adaptive, normatively reinforced summary function chain among many possible scripts.

Contradicting standard rational-choice theory, the authors assert that all goal-directed, maximization-compatible behaviors can be reframed as high-probability pattern completions in sufficiently enculturated actors, and that preferences are always endogenous, arising via consolidation of cultural and experiential information rather than exogenous parameterization.

This claim, if accepted, directly challenges the necessity of reward-maximization as a primitive for social modeling and multi-agent artificial intelligence, and instead grounds robust, goal-directed behavior in cultural pattern completion—a form of 'programmable rationality' through social training.

Implications for Cognitive Science, Sociology, and AI

This society-first, pattern completion model has several significant implications:

• Simulation of Social Systems: Population-level modeling can be achieved by initializing agents with culturally rich LLMs, allowing counterfactual and policy analyses that do not require bootstrapping from individualized reward specifications. This intersects with generative agent-based modeling and multi-agent LLM societies.
• Endogenous Preference Formation: Models can accommodate dynamic, history-dependent, and context-sensitive preference shifts, even in situations with abrupt cultural or institutional change.
• Norm Change and Social Tipping Points: Provides a quantitative theory of how collective memory, sanctioning dynamics, and bandwagon effects foster or resist change in norms, potentially informing policy design for large-scale social interventions or AI alignment strategies.
• Reframing Rationality in AI: Suggests that AI rationality, particularly in LLMs and multi-agent settings, should be understood as resulting from culturally transmitted epistemic norms rather than a universal, maximalist optimization principle. This impacts alignment, interpretability, and the design of human-compatible AI agents.

Theoretical and Practical Extensions

The theoretical commitments outlined invite rigorous empirical validation (e.g., via agent-based simulation platforms such as Concordia), particularly concerning normative drift, legacy norm retention, and the conditions under which explicit rational logics become consolidated as automated 'short path' completions. There is explicit potential for this formalism to inform the multi-actor modeling paradigms emerging in both AI safety and computational sociology, as well as to critically reevaluate approaches to value alignment, norm inference, and the explainability of agent choices in high-stakes environments.

Conclusion

The society-first predictive pattern completion theory elaborated in (2603.14050) offers a formally precise, computationally explicit account of normative cognition and social order. Its most significant contributions are the formal decentering of rational-choice theory, the unification of norm compliance, sanctioning, and cultural transmission via LLM-like architectures, and the explicit modeling of endogenous preference formation as a product of social pattern completion. The theoretical machinery provides a new lens for studying human cognition, social simulation, and the design and analysis of multi-agent AI, with broad implications for the development of culturally competent, interpretable machine agents and the empirical study of normativity in both natural and artificial societies.