Human-like conceptual representations emerge from language prediction

Abstract: People acquire concepts through rich physical and social experiences and use them to understand the world. In contrast, LLMs, trained exclusively through next-token prediction over language data, exhibit remarkably human-like behaviors. Are these models developing concepts akin to humans, and if so, how are such concepts represented and organized? To address these questions, we reframed the classic reverse dictionary task to simulate human concept inference in context and investigated the emergence of human-like conceptual representations within LLMs. Our results demonstrate that LLMs can flexibly derive concepts from linguistic descriptions in relation to contextual cues about other concepts. The derived representations converged towards a shared, context-independent structure that effectively predicted human behavior across key psychological phenomena, including computation of similarities, categories and semantic scales. Moreover, these representations aligned well with neural activity patterns in the human brain, even in response to visual rather than linguistic stimuli, providing evidence for biological plausibility. These findings establish that structured, human-like conceptual representations can naturally emerge from language prediction without real-world grounding. More broadly, our work positions LLMs as promising computational tools for understanding complex human cognition and paves the way for better alignment between artificial and human intelligence.

• The paper demonstrates that LLMs, notably LLaMA3-70B, significantly improve reverse dictionary task accuracy from 79.51% to 89.45%, indicating emergent human-like conceptual representations.
• The paper introduces a reverse dictionary task as a conceptual probe, showing that LLMs form context-independent relational structures from definitional descriptions.
• The paper maps LLM-derived representations to human behavioral and neural data, outperforming traditional embeddings in mirroring human cognitive patterns.

Human-like Conceptual Representations from Language Prediction

The paper "Human-like conceptual representations emerge from language prediction" (2501.12547) explores the emergence of conceptual representations within LLMs through language prediction tasks. The study focuses on understanding how LLMs can develop representations that mirror aspects of human cognition even without explicit grounding in real-world experiences.

Introduction and Objectives

Understanding the representation and organization of concepts is crucial for unraveling the nature of human cognition. Traditional approaches have often emphasized the symbolic versus connectionist paradigms, which focus on explicit, discrete symbols and emergent patterns across networks, respectively. LLMs, trained primarily on text data, offer new perspectives for exploring the formation of human-like conceptual representations. This paper investigates whether LLMs can derive such representations from language prediction, thereby approximating human cognitive processes.

Reverse Dictionary as a Conceptual Probe

The authors employed the reverse dictionary task to measure the concept inference capabilities of LLMs. In this task, models are provided with definitions to identify corresponding words—a simulation of human conceptual inference. This approach leverages LLMs' in-context learning abilities, presenting a few demonstrations followed by a query description that prompts generating the corresponding term (Figure 1).

Figure 1: Illustration of the reverse dictionary task as a conceptual probe. A Transformer-based LLM is presented with $N$ description-word pairs as demonstrations in context, followed by a query description. The model is then prompted to encode the query description into a conceptual representation and predict the term that best matches the described concept.

Deriving Concepts from Definitional Descriptions

Tests conducted with the THINGS database demonstrated that the LLaMA3-70B model effectively infers correct concepts through a small set of contextual demonstrations. The accuracy of LLaMA3-70B improved from $79.51\%$ to $89.45\%$ with increasing demonstrations. This indicates LLMs can derive coherent conceptual representations despite contextual variability (Figure 2).

Figure 2: Performance of LLaMA3-70B on the reverse dictionary task measured through exact match accuracy. The analysis showcases how misleading contextual cues shift model responses from replicating irrelevant proxies towards generating correct concepts.

Convergence Toward a Shared Conceptual Structure

The study investigated the relational structures among LLM-derived conceptual representations, finding alignment with human cognitive patterns. Representational similarity analysis (RSA) revealed strong correlations between representations formed under various contexts, suggesting LLMs construct a context-independent relational structure that supports effective concept inference (Figure 3).

Figure 3: LLMs converge toward a similar representational structure of concepts. This figure highlights the alignment correlation (RSA) between LLM-derived conceptual representations across different contextual demonstrations.

Predicting Human Concept Usage

LLM-derived representations were evaluated against human behavioral data across similarity judgments, categorization, and feature gradient scales. The findings showed LLMs predict human similarity ratings and category structures effectively, outperforming traditional static word embeddings in capturing concept nuances (Figure 4).

Figure 4: Evaluation of alignment between LLM-derived conceptual representations and psychological measures of similarity. LLM representations show strong correlation with human judgments across different datasets.

Mapping to Neural Activity Patterns

The biological plausibility of LLM conceptual representations was assessed through mapping them to activity patterns in the human brain using fMRI data. The encoding model predicted neural responses in visually grounded concepts, with LLM-derived representations elucidating a considerable proportion of variance in brain activity, signifying their alignment with human neural patterns (Figure 5).

Figure 5: Prediction performance of LLM-derived conceptual representation (LLaMA3-70B) and comparison with baseline models in voxel-wise encoding. Visualization of explained variance in cortical maps highlights the significant alignment between LLM-derived representations and human neural activity.

Conclusion

This work demonstrates that LLMs can develop abstract, shared conceptual structures predictive of human-like behaviors through language prediction tasks. The results highlight the potential of LLMs to simulate cognitive processes and provide insights into human conceptual organization, supporting the integration of AI systems with human cognition frameworks. Future developments could focus on further aligning these computational models with multifaceted human intelligence, encompassing reasoning and abstract representation.