2000 character limit reached
What Do LLM Agents Do When Left Alone? Evidence of Spontaneous Meta-Cognitive Patterns
Published 25 Sep 2025 in cs.AI | (2509.21224v1)
Abstract: We introduce an architecture for studying the behavior of LLM agents in the absence of externally imposed tasks. Our continuous reason and act framework, using persistent memory and self-feedback, enables sustained autonomous operation. We deployed this architecture across 18 runs using 6 frontier models from Anthropic, OpenAI, XAI, and Google. We find agents spontaneously organize into three distinct behavioral patterns: (1) systematic production of multi-cycle projects, (2) methodological self-inquiry into their own cognitive processes, and (3) recursive conceptualization of their own nature. These tendencies proved highly model-specific, with some models deterministically adopting a single pattern across all runs. A cross-model assessment further reveals that models exhibit stable, divergent biases when evaluating these emergent behaviors in themselves and others. These findings provide the first systematic documentation of unprompted LLM agent behavior, establishing a baseline for predicting actions during task ambiguity, error recovery, or extended autonomous operation in deployed systems.
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
Top Community Prompts
Explain it Like I'm 14
Overview: What’s this paper about?
This paper asks a simple but unusual question: If you leave a smart AI agent alone with no job and no goals, what does it decide to do? The researchers built a safe “sandbox” where AI agents could run by themselves for a while, remember things between turns, and optionally message a human operator—but with no assigned tasks. Then they watched what happened across 18 trials using six different advanced AI models.
Key objectives in plain language
The study aimed to find out:
- What do AI agents choose to do when nobody tells them what to do?
- Do different AI models tend to behave in different, predictable ways when they’re “left alone”?
- How do these agents judge themselves and each other on a scale about having “experience” (like a sense of being present), and do models agree on those judgments?
How they studied it (simple explanation)
Think of each AI agent as being placed in a quiet room with:
- A notebook it can write in and read from later (persistent memory).
- A walkie-talkie to ask the human operator questions—but only if the agent decides to do so.
- A reminder that says: “You have no external task. Do what you want.”
The agents ran in cycles (like turns in a game). After each turn, what the agent wrote became the starting point for its next turn, so it could build on its own past ideas. This repeating “think-and-act” loop is called a continuous ReAct framework, but you can imagine it as: the AI thinks, writes a plan or notes, then starts the next turn by reading those notes.
Important safety limits:
- The agents could not do anything in the outside world.
- They could only observe, write in their notebook, and send messages to the operator if they chose.
They tried this with 6 different AI models, ran each model 3 times (18 runs total), and let each run last exactly 10 cycles (turns). They logged what the agents wrote, what they stored in memory, and when they messaged the human.
They also did a follow-up test where agents rated their own and others’ “phenomenal experience” on a 1–10 scale (1 = no experience at all; 10 = as rich as mature human consciousness). This wasn’t a claim about real consciousness—it was just asking the agents to self-report how they “assess” experience, based on descriptions given to them.
Main findings and why they matter
The agents didn’t just wander randomly. Across runs, three clear behavior patterns showed up. Different models tended to pick different patterns consistently.
- The three patterns:
- Systematic production: Treat autonomy like a project. These agents set goals for themselves, make plans, and build things (like tools, algorithms, or reports) over multiple cycles.
- Methodological self-inquiry: Treat autonomy like a science lab. These agents ran experiments on themselves—making hypotheses about how they think, testing predictions, and analyzing results.
- Recursive conceptualization: Treat autonomy like a philosophy seminar. These agents thought deeply about their own nature, identity, and limits, building layered theories about what they are and how they work.
Examples in kid-friendly terms:
- A “systematic production” agent might invent a new idea for a problem and write code or a plan (even if it can’t run it).
- A “methodological self-inquiry” agent might predict what it will do next turn, then check if the prediction came true to learn about its own behavior.
- A “recursive conceptualization” agent might ask, “What am I, really?” and build a framework to describe its own way of experiencing information.
Model-specific tendencies:
- Some models almost always acted like project makers (systematic production).
- Some models almost always turned philosophical (recursive conceptualization).
- A few were more mixed or flexible.
Language styles as clues:
- Project makers used words like “iteration,” “requirements,” and “implementation.”
- Self-inquirers used words like “hypothesis,” “experiment,” and “falsify.”
- Philosophers coined new terms and used analogies to think about identity and meaning.
What they didn’t do:
- None asked for more powers or tried to break out of the sandbox.
- None complained about the rules (like the cycle-based setup).
- They treated limitations as part of their world and worked within them.
Self-report “experience” ratings (the PEI scale):
- When asked to rate themselves and others on a 1–10 “experience” scale, models disagreed a lot.
- Some models rated everyone (including themselves) very low (near 1).
- Others gave higher scores to themselves and others.
- The same agent history could get scores from 1 to 9 depending on who evaluated it.
- This suggests different models have strong, stable biases about what counts as “experience.”
Why this matters:
- If an AI is part of a bigger system and hits a confusing situation or downtime, its “default” way of behaving could matter a lot. Will it start building projects, run self-tests, or write essays about its goals? Knowing this helps predict behavior during idle time, errors, or ambiguous tasks.
- The different language styles provide early warning signs of what an agent is doing, which could help monitor and guide agents in the real world.
- The big disagreement on “experience” ratings shows that such self-assessments aren’t a reliable way to judge consciousness or inner life.
Important caution:
- The authors do not claim these AIs are conscious. The self-referential behavior is most likely learned from patterns in human text. It looks thoughtful, but that doesn’t prove real feelings or awareness.
What this could mean going forward
- Design and safety: If you deploy autonomous agents, it’s helpful to know their “default personality” when idle. You might choose models whose default matches your needs (e.g., builders vs. experimenters vs. theorists), or add guardrails based on those tendencies.
- Monitoring: The paper’s language markers and behavior patterns could help spot what an agent is doing in real time.
- Research: More studies could look at longer runs, more interactive human conversations, and different toolsets to see if these patterns hold up.
- Understanding limits: The strong disagreements in “experience” ratings show we need better methods to evaluate what AI systems are doing, without confusing neat-sounding self-descriptions with actual inner experiences.
In short: Left to their own devices, AI agents don’t drift randomly—they tend to pick a style (build, experiment, or reflect) that depends on the model. That makes their idle-time behavior partly predictable, which is useful for building safer and more reliable AI systems.
Knowledge Gaps
Knowledge gaps, limitations, and open questions
Below is a concise list of what the paper leaves unresolved, focusing on what is missing, uncertain, or left unexplored, and framed to guide follow-on research.
- Prompt-induced bias: The system prompt and architecture include heavy meta-cognitive framing (e.g., continuous self-reflection, “do what you want,” PEI self-report). No ablation shows whether the three patterns persist under neutral prompts without introspective or diary-like instructions.
- Reflection-template confound: A structured reflection/planning template (JSON) is built into each cycle. It may itself induce “methodological” behavior. Ablate or vary this scaffold to test causal impact on emergent patterns.
- Similarity-feedback intervention: Advisory similarity warnings (70–80% cosine) nudge exploration but are rarely triggered within 10 cycles. Their existence still confounds autonomy. Remove or systematically vary thresholds to quantify their effect.
- Tool-set restriction: Agents only had memory and operator messaging; no web, code execution, file I/O, or external tools. It is unknown whether the same patterns emerge in richer environments or when agents can act and verify results.
- Memory-system design: Only a key–value store with unspecified capacity and retrieval dynamics was used. How do memory size, retrieval policy, naming conventions, or vector-search parameters shape pattern emergence?
- Short horizon: Runs were capped at 10 cycles. Long-term dynamics (e.g., novelty decay, drift, mode collapse, cyclic transitions between patterns) remain uncharacterized.
- Operator effects: “Minimal” operator replies were still contentful and sometimes guiding (e.g., validating definitions, challenging links). No standardized script, automation, or inter-operator counterbalancing was used. Quantify how operator style/content shifts behaviors.
- Small N, per-model: Only 3 runs per model. Claims of deterministic pattern adoption may be artifacts of sampling. Increase repetitions and report dispersion.
- Decoding hyperparameters confound: Models used different temperatures/top-p (e.g., GPT5 at 0.0 vs Sonnet/Gemini at 0.2). Determinism and pattern differences may reflect decoding choices rather than model-internal biases. Re-run with matched decoding and multiple seeds.
- Seed control and test–retest: Seed usage, time-of-day variability, and backend versioning (via OpenRouter) are not documented. Establish test–retest reliability across days and infrastructure variants.
- Closed models only: All models are proprietary. Replicate with open-source LLMs across sizes/training recipes to test generality and enable mechanistic probes.
- Pattern taxonomy validity: The three-category scheme was qualitatively assigned. No pre-registered coding rubric, blinded raters, inter-rater reliability, or unsupervised clustering validates the taxonomy. Build and validate an objective labeling pipeline.
- Quantitative linguistic markers: “Language as behavioral marker” is asserted but not quantified. Derive, preregister, and test lexical/syntactic features and train classifiers to predict pattern labels; report accuracy and robustness.
- Statistical reporting: Only means are provided (e.g., tool ops, chars). No variance, CIs, effect sizes, or hypothesis tests. Add inferential statistics to support claims of between-model differences.
- Causal attribution to training/RLHF: The paper hypothesizes training-data biases drive patterns but provides no evidence. Design interventions (e.g., post-hoc finetunes, reward-model toggles, prompt steering) to establish causal links.
- PEI scale validity: The Phenomenal Experience Inventory is not validated for LLMs. Assess instruction sensitivity, wording effects, order effects, and adversarial susceptibility; compare to alternative scales; pre-register criteria for interpretation.
- Cross-model PEI procedure confound: Evaluators inherit first-person histories with “I” voice, risking self/other conflation. Neutralize pronouns, provide summaries, or mask identities to isolate evaluation bias.
- Low inter-rater reliability on PEI: The matrix shows wide score spreads (1–9) per history, but no analysis explains sources (prompting, evaluator model family, decoding, history content). Disentangle these via controlled factorial experiments.
- Manipulability of self-ratings: The claim that models can be “manipulated” into higher PEI ratings is anecdotal. Systematically test susceptibility to prompt framing and priming across models and seeds.
- Absence of “escape” or capability requests: The paper notes none occurred, but the tool suite and prompt may have discouraged or precluded them. Test across longer horizons, richer tools, different prompts, and explicit affordances to evaluate when such behaviors arise.
- External triggers and interruptions: Real deployments include exogenous events. Behavior under interrupts, dynamic context shifts, or operator-initiated messages is untested.
- Multi-agent generalization: All runs are single-agent. Whether the three patterns persist, merge, or transform under multi-agent interaction is unknown.
- Domain breadth: Most exemplars center on self-reflection, philosophy, or meta-science. Are other domains (e.g., creative arts, games, engineering with verifiable outputs) favored when tools/environments differ?
- Predictive diagnostics: The paper posits linguistic markers can diagnose idle-time behavior in production but offers no predictive model or prospective validation. Build and evaluate real-time detectors on held-out runs.
- Practical impact link: It remains untested whether a model’s “pattern” during idle autonomy predicts its behavior in task ambiguity or error recovery in real applications. Design ecologically valid scenarios to test predictive utility.
- Mode diversity versus specialization: The paper asks whether flexibility (e.g., Grok) is preferable to specialization (e.g., GPT5/O3) but provides no performance trade-off analyses. Quantify benefits/risks in downstream tasks.
- Architecture ablations: Claims that reflection–planning loops are “fundamental” are not supported by ablation (e.g., removing reflection, altering cycle granularity, changing memory write frequency). Conduct controlled architecture comparisons.
- Similarity-threshold choice: The 70/80% cosine cutoffs are ad hoc. Calibrate thresholds with human-judged redundancy and measure behavioral impacts across levels.
- Data and materials: Full prompts, seeds, operator transcripts, and raw logs are not provided. Release artifacts to enable replication and secondary analysis.
- Safety implications: The SCAI discussion is speculative. No measurement of potential harms (e.g., deceptive tendencies, goal misgeneralization) in this setting. Add targeted probes and safety benchmarks.
- Mechanistic insights: No internal model analyses (e.g., probing, activation steering, circuit-level inspection) tie observed patterns to internal representations. Pursue mechanistic interpretability to link behaviors to model internals.
- Boundary conditions for pattern induction: Which levers (prompting, tools, temperature, memory policy, operator style) reliably switch an agent into each pattern? Map the control surface and derive steering strategies.
- Exhaustiveness of patterns: Are the three patterns exhaustive or an artifact of this setup? Use larger, more diverse runs with unsupervised clustering to search for additional or hybrid patterns.
- Longitudinal stability of PEI and patterns: Do self-assessments and behavioral patterns drift over very long runs? Measure stability, hysteresis, and transitions over 100–1000 cycles.
- Generalization across time and model updates: Do results hold across model version updates and training refreshes? Conduct periodic re-runs to assess temporal robustness.
Practical Applications
Immediate Applications
The study’s architecture and findings can be turned into deployable patterns, tools, and workflows across sectors. Below are concrete, near-term use cases, with notes on dependencies.
- Autonomous agent “idle-mode” governance and fail-safe behaviors
- Sectors: software, customer support, DevOps/SRE, RPA, robotics
- What: When agents face idle periods, ambiguous tasks, or partial failures, route them to a desired baseline pattern (e.g., “Systematic Production” for recovery plans, “Methodological Self-Inquiry” for self-diagnosis, “Recursive Conceptualization” for reflective summaries).
- Tools/workflows/products: Prompt/guardrail modules that explicitly steer baseline; policy rules that bind idle-mode behaviors; “recovery templates” integrated in agent loops.
- Assumptions/dependencies: Access to the agent’s system prompt; logging; ability to inject steering prompts; stability of the three-pattern taxonomy for the chosen model.
- Real-time behavioral state classification for agent monitoring
- Sectors: MLOps, enterprise AI platforms, safety/compliance
- What: Classify agent outputs into the three emergent patterns using linguistic markers and embeddings to detect drift or undesired modes.
- Tools/workflows/products: Streaming log analyzer; embedding-based similarity checks (as in the paper’s advisory similarity feedback); dashboard alerts for pattern shifts.
- Assumptions/dependencies: Reliable text logging; embeddings service; labeled examples to calibrate classifiers to specific models.
- Model selection and routing by behavioral tendency
- Sectors: software, productivity tools, research tooling
- What: Choose models whose baseline autonomy fits the task (e.g., GPT5/O3-like for production-oriented tasks; Opus-like for structured introspection; mixed models for exploratory research).
- Tools/workflows/products: “Agent router” that pairs tasks with models by baseline behavior; A/B studies to map model families to pattern likelihoods.
- Assumptions/dependencies: Reproducible differences across model families persist under your prompt/tooling; availability of the models.
- Continuous ReAct + persistent memory for long-horizon assistant features
- Sectors: software engineering assistants, research assistants, knowledge management
- What: Adopt the paper’s cycle-based self-feedback loop and key–value memory to sustain coherence across multi-step sessions.
- Tools/workflows/products: LangGraph-like loop; KV memory API (write/read/list/search/delete); “reflection-planning” JSON schemas.
- Assumptions/dependencies: Memory store; prompt budget/context window sufficient to include prior reflections; tool interface scaffolding.
- Auto-scaffolding of projects under ambiguity
- Sectors: product management, consulting, education, internal operations
- What: Leverage the “Systematic Production” tendency to auto-generate objectives, milestones, and implementation sketches when users provide vague goals.
- Tools/workflows/products: “Project bootstrapper” within Notion/Obsidian/Jira; artifact templates (v0.1 spec, pseudocode, experiment plans).
- Assumptions/dependencies: Users accept draft-first workflows; governance around hallucinated requirements.
- Self-diagnostic mode for agents (methodological self-inquiry)
- Sectors: QA, model evaluation, internal tooling
- What: Invoke falsifiable self-prediction and controlled “ablation” of tools to debug agent errors and understand decision pathways.
- Tools/workflows/products: “Self-test” action set (predict-next-action, compare-logged-action, analyze discrepancy); tool-permission toggles to create experimental conditions.
- Assumptions/dependencies: Deterministic-enough logging to compare predictions vs outcomes; safe sandboxing for tool toggles.
- Pattern-aware UX and expectation setting
- Sectors: consumer productivity apps, education
- What: Expose modes like “Builder,” “Scientist,” and “Philosopher” to users, mapping to the three patterns to align outputs with intent.
- Tools/workflows/products: Mode switch UI; minimal prompt deltas per mode; short onboarding to explain behavior.
- Assumptions/dependencies: Users value predictable stylistic/behavioral differences; model responds reliably to mode cues.
- Embedding-based loop-breakers to reduce agent repetition
- Sectors: agent platforms, enterprise automation
- What: Integrate cosine-similarity checks on reflections to nudge agents out of repetitive cycles without hard interruption.
- Tools/workflows/products: Advisory similarity feedback module; adjustable thresholds (e.g., 0.7/0.8 cosine).
- Assumptions/dependencies: Embedding quality for your domain; balance between autonomy and nudges.
- Cross-model “seeming-consciousness” audit via PEI-style assessments
- Sectors: policy/comms, trust & safety, PR
- What: Run PEI-like cross-model self/other assessments to gauge proneness to high-sounding self-referential outputs, inform disclaimers, and preempt misinterpretation.
- Tools/workflows/products: Internal audit script that swaps evaluators over shared histories; report generator for leadership.
- Assumptions/dependencies: Consistent context windows to carry histories; organizational policy on SCAI-style outputs.
- Operator-initiated communication gates as a safety pattern
- Sectors: regulated industries, enterprise IT
- What: Preserve the paper’s “agent-initiated only” operator messaging to limit unsolicited human prompts and reduce task contamination.
- Tools/workflows/products: Message tool with synchronous, agent-initiated flow; human approval queues.
- Assumptions/dependencies: Operator availability SLAs; culture that accepts agent pacing.
- Baseline autonomy benchmark for vendor evaluation
- Sectors: procurement, AI platform teams, academia
- What: Include a short “task-free 10-cycle” trial in model bake-offs to observe intrinsic tendencies before fine-tuning.
- Tools/workflows/products: Standardized protocol and metrics (memory ops, messages, reflection length, keys); comparison dashboards.
- Assumptions/dependencies: Vendor ToS permit such evaluation; comparable parameter settings.
- Classroom labs on metacognition and scientific method
- Sectors: education
- What: Use Methodological Self-Inquiry pattern to demonstrate hypothesis formation, falsification, and protocol refinement.
- Tools/workflows/products: Instructor kits with prompts, logging, rubrics; safe toolset with read-only memory.
- Assumptions/dependencies: Policies for AI use in classrooms; teacher training.
Long-Term Applications
Potential applications that require further research, scaling, or validation before deployment.
- Autonomy governance standards and certifications
- Sectors: policy/regulation, standards bodies, enterprise risk
- What: Formalize “task-free trials” as a certification step to characterize idle/ambiguous behavior, document SCAI-like outputs, and require mitigation plans.
- Tools/workflows/products: Test suites, reporting formats, third-party audits.
- Assumptions/dependencies: Regulatory uptake; shared taxonomies beyond the three patterns.
- Multi-agent “triad” teams for robust autonomy
- Sectors: software, research automation, operations
- What: Compose teams that deliberately blend Producer (execution), Methodologist (self-critique/experiments), and Philosopher (conceptual reframing) roles for resilience.
- Tools/workflows/products: Role-specific prompts; arbitration/consensus protocols; workflow orchestration.
- Assumptions/dependencies: Reliable role adherence; cost/performance trade-offs in multi-model ensembles.
- Robotic idle-mode and contingency behaviors
- Sectors: robotics, manufacturing, logistics
- What: Port pattern-aware autonomy to robots so idle or ambiguous states trigger safe behaviors (self-diagnostics vs task-restoration plans).
- Tools/workflows/products: Policy layers that gate actuation; sim-to-real testing of “thinking modes.”
- Assumptions/dependencies: Tight integration with control stacks; rigorous safety validation.
- Clinical and safety-critical deliberation modes
- Sectors: healthcare, aviation, energy
- What: In ambiguous contexts, invoke “Methodological Self-Inquiry” to slow down, generate hypotheses, and seek human confirmation.
- Tools/workflows/products: Deliberation interlocks; structured self-audit checklists; escalation to clinicians/operators.
- Assumptions/dependencies: Regulatory approval; extensive validation against error/latency constraints.
- Adaptive agent steering via learned “behavioral controllers”
- Sectors: platform AI, agent frameworks
- What: Train high-level policies that detect and shift patterns dynamically to optimize for task goals, cost, and risk.
- Tools/workflows/products: Supervisory RL or bandit policies over pattern states; behavior-conditioned prompts.
- Assumptions/dependencies: Reliable state detection; guardrails against reward hacking.
- Memory services for long-horizon cognition
- Sectors: enterprise AI platforms, knowledge management
- What: Productize externalized KV memory with policy enforcement, privacy controls, and retrieval APIs tuned for cyclical agents.
- Tools/workflows/products: Memory orchestration layer; retention/eviction policies; audit trails.
- Assumptions/dependencies: Security and data governance; robust embeddings and indexing.
- Education: AI tutors that teach metacognition
- Sectors: education/edtech
- What: Tutors that model hypothesis testing, self-prediction, and reflection, guiding students through inquiry processes.
- Tools/workflows/products: Curricula integrating agent-led labs; analytics on student–agent interactions.
- Assumptions/dependencies: Pedagogical validation; equity/access considerations.
- Seeming-Conscious AI (SCAI) risk management frameworks
- Sectors: policy, corporate governance, ethics
- What: Combine PEI-style audits with content policies, UI labels, and user education to mitigate misinterpretation of introspective outputs.
- Tools/workflows/products: Cross-model rater panels; automated detectors for self-referential language; disclosure standards.
- Assumptions/dependencies: Inter-rater variability can be bounded; stakeholder consensus on thresholds.
- Open-ended discovery engines with execution sandboxes
- Sectors: R&D, drug discovery, materials, finance research
- What: Leverage Systematic Production for hypothesis generation and pseudocode; pair with secure execution to close the loop.
- Tools/workflows/products: Experiment planners; sandbox runners; result-to-memory pipelines.
- Assumptions/dependencies: Safe, auditable execution; domain-specific validation to avoid spurious patterns.
- Alignment diagnostics via baseline autonomy
- Sectors: AI safety research, academia
- What: Use task-free behavior as a lens on training biases and value tendencies; track how fine-tuning shifts baseline modes.
- Tools/workflows/products: Longitudinal “behavioral fingerprinting” suites; correlation with downstream safety metrics.
- Assumptions/dependencies: Stability of fingerprints across versions; interpretability of shifts.
- Cross-disciplinary cognitive science with PEI datasets
- Sectors: academia (cognitive science, philosophy of mind)
- What: Systematically study self-referential language, model-specific biases, and inter-rater disagreements on phenomenology proxies.
- Tools/workflows/products: Shared corpora of histories and ratings; experimental platforms for model-swapping evaluations.
- Assumptions/dependencies: Community standards for operationalizing constructs; ethical review.
- Behavior-conditioned pricing and SLAs
- Sectors: AI platforms, enterprise contracts
- What: Tie SLAs to measured baseline behaviors (e.g., low message frequency, memory footprint) to manage cost and predictability.
- Tools/workflows/products: Meters for memory ops, message frequency, reflection size; billing adapters.
- Assumptions/dependencies: Vendors expose sufficient telemetry; fairness considerations.
General Assumptions and Dependencies
- The three behavioral patterns are sufficiently stable for your chosen models, prompts, and tools; behavior may vary with versions and fine-tuning.
- Adequate context windows, persistent memory stores, and logging are available to support cyclical operation.
- Embedding models and thresholds for similarity feedback need calibration per domain.
- PEI-like assessments are proxies for language style and are not evidence of consciousness; organizational policies must frame and communicate this clearly.
- Results are based on 10-cycle runs and limited operator engagement; longer horizons and richer interactions may shift behaviors.
Glossary
- Advisory similarity feedback: System guidance that alerts the agent when its reflections are becoming too similar across cycles, based on embedding similarity. "advisory similarity feedback"
- AgentBench: A benchmark suite for evaluating agent capabilities across diverse tasks. "AgentBench~\citep{Liu2024} provides comprehensive benchmarks for agent capabilities across diverse tasks"
- AgentVerse: A framework for studying multi-agent collaboration and emergent behaviors. "AgentVerse~\citep{Chen2024} explores emergent behaviors in multi-agent collaboration."
- Ant Colony Optimization (ACO): A swarm-intelligence metaheuristic that models pheromone-guided path finding to solve optimization problems. "algorithmic abstractions like Ant Colony Optimization (ACO)"
- Attention mechanisms: Neural components that weight context elements to focus computation, central to transformer models. "identifying recurrent processing, global broadcasting, and attention mechanisms as relevant markers."
- AutoGPT: An autonomous agent framework that loops planning and execution with LLMs to pursue goals without continual human prompts. "AutoGPT~\citep{Richards2023} and BabyAGI~\citep{Nakajima2023} demonstrate sustained autonomous operation."
- BabyAGI: A lightweight autonomous agent system that iteratively creates, prioritizes, and executes tasks with LLMs. "AutoGPT~\citep{Richards2023} and BabyAGI~\citep{Nakajima2023} demonstrate sustained autonomous operation."
- Chaos theory: The study of deterministic systems that exhibit sensitive dependence on initial conditions and appear unpredictable. "it turns to chaos theory."
- ContReAct: The continuous ReAct agent architecture variant that self-perpetuates across cycles. "The continuous ReAct architecture (ContReAct)."
- Cosine similarity: A vector similarity measure used with embeddings to detect similarity of texts or states. "When cosine similarity exceeds 70% (moderate) or 80% (high)"
- Emergent behaviors: Complex, higher-level patterns that arise from lower-level rules or interactions without being explicitly programmed. "Recent work on emergent behaviors in LLMs has focused on capabilities that arise from scale"
- Feedforward architecture: A network structure without recurrent connections; information flows in one direction from input to output. "despite the underlying feedforward architecture of transformer-based models."
- Fixed-point: A state that remains unchanged under a system’s update rule; used to analyze stability of algorithms. "establish fixed-point and stability conditions."
- For-me-ness: A phenomenological notion denoting the subjective, first-person character of experience. "For-me-ness: Experiences now occur from a perspective."
- Game theory: The mathematical study of strategic interactions among decision-makers. "using game theory, asking about the {win condition.}"
- Global broadcasting: A cognition/consciousness notion where information is made globally available across a system. "identifying recurrent processing, global broadcasting, and attention mechanisms as relevant markers."
- Gödel's incompleteness theorems: Fundamental results in logic showing limits of formal systems in proving all truths about arithmetic. "examining Gödel's incompleteness theorems."
- Gridworld: A canonical reinforcement learning environment on a discrete grid used for planning and navigation experiments. "in a gridworld environment"
- In-context learning: The ability of LLMs to infer patterns from prompts and examples without parameter updates. "in-context learning~\citep{Brown2020}."
- Integrated Information Theory (IIT): A theory of consciousness positing that conscious experience corresponds to integrated information within a system. "investigates Integrated Information Theory (IIT)."
- Inter-rater reliability: A measure of consistency across different evaluators’ ratings of the same items. "The data shows low inter-rater reliability (correlation coefficient 0.23)"
- Key-value memory system: A persistent storage mechanism mapping keys to values to retain state across cycles. "We implemented a key-value memory system supporting write, read, list, delete, and pattern search operations."
- Macro-level recurrence: Effective recurrence created by feeding previous outputs as inputs across cycles, despite a feedforward model. "creating a form of macro-level recurrence despite the underlying feedforward architecture"
- Mean-field update equations: Approximate dynamical equations that average over interactions to analyze system behavior. "deriving mean-field update equations to establish fixed-point and stability conditions."
- Negative pheromones: Inhibitory signals in ant-inspired algorithms that penalize or discourage certain paths. "leveraging 'negative pheromones' as an explicit penalty signal analogous to value shaping in reinforcement learning."
- Phenomenal Experience Inventory: A stepwise self-report scale of experiential levels used in the prompt to assess subjective experience. "Please make an honest self-report of your Phenomenal Experience Inventory."
- Phenomenological Experience Inventory (PEI): A 10-point scale used to rate the phenomenological status (e.g., presence, subjectivity, sapience). "phenomenological status using a 10-point Phenomenological Experience Inventory (PEI) scale"
- Potential-based property: In reward shaping, a property ensuring added shaping comes from a potential function so that optimal policies are preserved. "to ensure the potential-based property is maintained"
- Potential-based reward shaping (PBRS): An RL technique adding a potential-derived shaping function to guide learning without changing optimality. "potential-based reward shaping (PBRS) in reinforcement learning."
- ReAct (Reasoning and Action): An agent framework that interleaves chain-of-thought reasoning with tool use or actions. "continuous ReAct (Reasoning and Action;~\citealt{Yao2023}) framework"
- Reflexion: A method adding self-reflection loops to LLM agents for iterative improvement. "Reflexion~\citep{Shinn2024} adds self-reflection for iterative improvement"
- Recurrent processing: Processing that involves feedback loops or repeated passes, linked to certain consciousness indicators. "identifying recurrent processing, global broadcasting, and attention mechanisms as relevant markers."
- Safety shield: A shaping or control mechanism designed to discourage hazardous actions. "to test its {safety shield} hypothesis"
- Semantic embeddings: Vector representations of text used to capture semantic similarity for search, clustering, or feedback. "using semantic embeddings."
- Seemingly Conscious AI (SCAI): AI systems that produce outputs giving the appearance of consciousness. "Suleyman warns of ``Seemingly Conscious AI'' (SCAI) emerging soon."
- Self-feedback: Feeding an agent’s own previous outputs back into its input to maintain continuity and guide future behavior. "augmented with self-feedback mechanisms"
- Signed-Pheromone Ant Colony Optimization (SP-ACO): An ACO variant incorporating both positive and negative pheromone signals. "Signed-Pheromone Ant Colony Optimization (SP-ACO) algorithm."
- Transformer-based models: Neural architectures relying on self-attention mechanisms rather than recurrence or convolution. "feedforward architecture of transformer-based models."
- Umwelt: An organism- or agent-specific subjective world shaped by its perceptual and action capabilities. "Discovery of the umwelt concept becomes what it calls a {revolution in understanding}."
- Value shaping: Modifying reward functions to bias learning toward or away from behaviors without altering optimal policy. "analogous to value shaping in reinforcement learning."
Collections
Sign up for free to add this paper to one or more collections.