Great Linguistics: Theory & Computational Insight

• Great Linguistics is a research paradigm that combines generative linguistics, cognitive models, Bayesian methods, and LLMs to explain language structure and acquisition.
• It employs formal constructs like Merge and Agreement alongside empirical metrics, revealing data-efficient language learning and cross-linguistic predictions.
• Integrative approaches using Bayesian statistics and computational experiments enhance the replicability, interpretability, and scientific rigor of linguistic research.

Great Linguistics encompasses the contemporary convergence of formal generative theory, cognitive models, Bayesian statistics, and the computational paradigm instantiated by LLMs. It is marked by a dual commitment: to explanation grounded in explicit, interpretable theoretical constructs (as championed by generative linguistics) and to empirical validation by experimental, computational, and quantitative means. This research domain interrogates the nature of linguistic knowledge, the mechanisms of acquisition and processing, and the prospects for integrating engineering systems such as LLMs with the scientific study of human language.

1. The Generative Core and Explanatory Mandate

A foundational tenet of Great Linguistics is the indispensability of generative theory for explaining both what is possible in natural language and why. Generative linguistics investigates universal properties of language, proposing formal apparatuses—Merge, Agree, feature-checking, movement operations, binding domains—that yield explicit accounts of structural phenomena across typologically diverse languages. Central explanatory achievements include:

• Accounting for data-efficient language acquisition: Empirical findings show that children acquire fluent native language with exposure to only around $10^7$ tokens per year and manifest complex syntax between ages 3 and 5. In stark contrast, state-of-the-art LLMs are trained on $10^{11}$–$10^{12}$ tokens, with no evidence of matching the data-efficiency or rapid generalization observed in children.
• Elucidating the Poverty of the Stimulus (PoS): Human learners generalize beyond the information in their input, a phenomenon formalized in computational learning theory (CLT) as a trade-off between the class of concepts $C$, types of data $D$, and learning algorithm $A$, with provable sample complexity bounds. Learners with unconstrained hypothesis spaces require infeasible amounts of data or time; hence, innate structural constraints (Universal Grammar) are posited (Kodner et al., 2023).
• Enabling formal and cross-linguistic predictions: Theoretical constructs distinguish possible from impossible agreement systems and explain phenomena such as Principle A of binding theory, which specifies locality and c-command requirements for reflexive interpretation.

Generative linguistics, by providing both empirical predictions and concise, interpretable explanations, remains foundational for the scientific program in linguistics (Kodner et al., 2023).

2. LLMs in Theoretical and Experimental Linguistics

Great Linguistics recognizes large pre-trained LMs as both engineering feats and novel scientific tools. LMs, trained on large-scale next-token prediction objectives,

$P(w_t \mid w_{<t})$

and cross-entropy minimization, enable linguists to probe representational and generalization properties at scale. Key research findings include:

• Syntax: LMs trained autoregressively internalize aspects of syntactic structure, as shown by their performance on subject-verb agreement, hierarchical embeddings, and filler–gap dependencies. Probing studies recover part-of-speech, constituency, and relational information from internal representations.
• Semantics: Embeddings and context vectors encode semantic roles, entailment relations, and thematic fit, with probabilistic scoring aligning with psycholinguistic acceptability (Futrell et al., 28 Jan 2025).
• Morphology: Character-level and subword-based LMs capture morphological categories in distributed continua, supporting both categorical and gradient generalizations.
• Metalinguistic capacities: Contemporary LLMs (e.g., GPT-4 and OpenAI o1) can generate syntactic trees in the X-bar framework, state phonological rules, and express lambda-calculus based semantic analyses in response to novel prompts. This behavioral interpretability opens a new direction for investigating the internalization of linguistic theory by LMs (Beguš et al., 2023).

Table: Model Capabilities Identified in Metalinguistic Probing (Beguš et al., 2023)

Task	GPT-4 / o1 Result	GPT-3.5 Result
X-bar tree drawing	Largely correct, recursive	Hallucinates, errors
Phonological generalization	Rule-like abstraction	Frequent mistakes
Semantic analysis	Expresses lambda terms	Occasional confusion

Despite advances, LLMs rely on architectures and learning paradigms (e.g., back-propagation, large context windows) that are biologically implausible and demand far greater data than biological learners, reinforcing the need for independent cognitive theory (Kodner et al., 2023).

3. Integration of Bayesian Methods in Linguistic Inference

A prominent methodological development is the insertion of Bayesian statistics into the core of linguistic research. Bayesian regression and hierarchical models have supplanted or complemented frequentist paradigms due to increased software accessibility, growing epistemological demands (replicability crisis), and alignment with cognitive models.

Distinguishing features:

• Probability as degree of credence (not frequency); all unknowns are random variables with posterior distributions.
• Core inferential mechanism:

$$P(\theta \mid D) \propto P(D \mid \theta) \times P(\theta)$$

where $P(\theta)$ is the prior, $P(D \mid \theta)$ the likelihood, and the product yields the posterior.

• Rich articulation of uncertainty (posteriors, high-density intervals), straightforward incorporation of random-effect structures (phylogenetic, typological covariance), and flexible handling of non-Gaussian, bounded, and high-dimensional data.
• Bayesian hypothesis testing via Bayes factors, probability of direction (PD), regions of practical equivalence (ROPE), and posterior predictive checks.
• Software ecosystem includes Stan, brms, JAGS, and GUI tools like JASP, with rigorous workflow recommendations for reproducibility and sensitivity analysis (Levshina, 21 Sep 2025).

This methodological turn enables more nuanced and transparent inferences, especially valuable for integrating empirical data with theoretical constraints.

4. Unified Cognitive and Evolutionary Models

The prospect of a unified theory is exemplified by Bayesian construction grammar frameworks and evolutionary accounts positing language as an adaptation for intelligence display via sexual selection (Worden, 14 Aug 2025). Core elements:

• Cognitive computation is explicitly Bayesian: feature-structures (constructions) are unified through maximum-likelihood pattern matching, integrating phonology, syntax, semantics, and pragmatics into a single mechanism. Formally, unification corresponds to constructing $C = A \sqcup B$, representing information-minimizing subsumption across feature-structures.
• Language evolution is explained as a runaway effect of sexual selection for mind-reading ability and rapid-turn conversational prowess, providing both the unique species-specificity and the intensity of human linguistic capacity.
• Learning unfolds in two stages: slow, general inference for bootstrapping new constructions from sparse examples, followed by rapid, parallel unification in processing, with explicit pseudo-code workflows delineating each process.
• All layers—from Theory of Mind and societal cooperation to self-esteem and emotion—are unified within the Bayesian construction grammar architecture, supporting both communicative and sociocultural phenomena.

5. Limits, Controversies, and Research Frontiers

Great Linguistics is defined as much by its open questions as by its achievements. Principal debates and trajectories include:

• Data efficiency and inductive bias: LMs' reliance on massive corpora and opaque parameterizations motivates continued inquiry into innate constraints and hypothesis space design.
• Simulation vs. explanation: Engineering demonstrations (LM performance) are not sufficient to explain human cognition; "simulation is not duplication." Thus, explicit theory remains essential (Kodner et al., 2023).
• Interpretability and openness: Proprietary LLMs violate scientific standards of replicability; lack of transparency frustrates causal and mechanistic interpretation (Kodner et al., 2023).
• Interface of symbolic and gradient representation: The tension between rule/vocabulary models and distributed, usage-based generalizations drives research on hybrid architectures and probing techniques.
• Benchmarking and theory-driven evaluation: Advances in test suites (BLiMP, SyntaxGym, BabyLM) exemplify the need for theory-informed empirical research (Kodner et al., 2023, Futrell et al., 28 Jan 2025).
• Methodological rigor: Bayesian practices (prior elicitation, sensitivity analysis, reproducibility) and computational learning theory establish criteria for robust inference and model evaluation (Levshina, 21 Sep 2025).

Suggested directions include controlled-data experiments, multimodal grounding, mechanistic interpretability, and cross-linguistic generalization to under-represented languages.

6. Synthesis and the Prospect of Great Linguistics

In integrating generative theory, probabilistic inference, computational modeling, and cognitive-evolutionary hypotheses, Great Linguistics:

• Provides a principled account of the extraordinary generalization capabilities of human learners
• Formalizes all levels of linguistic structure, from phonology to pragmatics, within a unified, interpretable framework
• Leverages computational technologies as new laboratories for scientific inquiry, while recognizing their limits
• Implements rigorous quantitative methodologies rooted in Bayesian statistics and formal learning theory
• Maintains explanatory depth—requiring not only predictions but also mechanisms and cross-linguistic theory formulation
• Addresses phenomena of cognition, cooperation, and sociality as inherent consequences of the language faculty

This approach establishes Great Linguistics as both an empirical and explanatory science, poised to remain central to the study of language well into the 21st century (Kodner et al., 2023, Futrell et al., 28 Jan 2025, Levshina, 21 Sep 2025, Worden, 14 Aug 2025, Beguš et al., 2023).