Digital Red Queen: Adversarial Program Evolution

• Digital Red Queen is a self-play algorithm that evolves adversarial programs via LLM-guided mutation, simulating continuous adaptation in dynamic environments.
• The framework employs a quality-diversity search using MAP-Elites in Core War, optimizing performance through behavior descriptors and historical lineage.
• Empirical results show convergent evolution, reduced cyclic dominance, and enhanced robustness against static human-designed warriors.

Digital Red Queen (DRQ) is a minimal self-play algorithm for evolving adversarial programs, harnessing LLMs to realize continual adaptation in a dynamic optimization landscape. Unlike classical static evolutionary approaches, DRQ models the perpetual arms race seen in biological Red Queen phenomena, wherein each new solution must outperform a lineage of previous champions. The framework is instantiated in the Core War environment—a Turing-complete artificial life and cybersecurity sandbox—where assembly-like warriors compete for control of a virtual machine. DRQ demonstrates the emergence of increasingly general and convergent strategies, positioning itself as a paradigmatic method for adversarial program evolution and as a testbed for LLMs in open-ended domains (Kumar et al., 6 Jan 2026).

1. Formal Objective and Red Queen Dynamics

DRQ is defined by a recursive, self-referential optimization objective. Traditional static optimization evolves a program $w$ to maximize fitness against a fixed set of opponents $\{u_1,\dots,u_K\}$; DRQ instead evolves a new warrior $w_t$ at each round $t$ to defeat the expanding set of historical champions $\{w_0,\dots,w_{t-1}\}$:

$$w_t = \arg\max_{w} \mathbb{E}_{\text{seeds}}\left[\text{Fitness}\left(w; \{w_0,\ldots,w_{t-1}\}\right)\right]$$

Fitness is context-dependent and recalculated with each augmentation of history. In Core War, for $N$ warriors over $\mathcal T$ timesteps, fitness rewards both survival and elimination of opponents:

$$\text{Fitness}(w_i;\{w_j\}_{j\neq i}) = \sum_{\tau=1}^{\mathcal T} \frac{N}{\mathcal T} \frac{A^i_\tau}{\sum_j A^j_\tau}$$

where $A^i_\tau\in\{0,1\}$ denotes whether warrior $i$ is alive at time $\tau$. This shifting fitness landscape induces continual evolutionary pressure and adaptive dynamics, analogous to Red Queen processes in biological systems.

2. Self-Play Loop and Quality-Diversity Optimization

DRQ operationalizes these dynamics via an outer self-play loop coupled to an inner quality-diversity search (MAP-Elites). The core algorithm maintains a lineage of warriors, updating it as follows:

Input: initial warrior w0, total rounds T, per-round iterations I, history-length K Output: lineage [w0, w1, …, wT] history ← [w0] for t in 1..T: A ← initialize_archive(cells) for u in history: insert u into A at cell BD(u) targets ← last K elements of history for iter in 1..I: p ← sample_elite(A) c ← mutate_with_LLM_or_random(p) f, bd ← evaluate_fitness_and_descriptor(c, targets) A ← update_archive(A, c, bd, f) wt ← best_elite(A) append wt to history return history

Key hyperparameters include rounds $T\approx10$–$100$, iterations per round $I=1,000$, and history length $K\in\{1,3,10\}$. Behavior descriptor cells are defined on total spawned threads and total memory coverage, both discretized log-scale. Larger $K$ reduces the prevalence of cyclic dominance (e.g., rock–paper–scissors cycles), yielding more robustly general strategies.

3. Warrior Representation and LLM-Guided Mutation

Each Core War warrior is represented as a Redcode assembly string, capped at 100 instructions and annotated with ORG/END directives. The instruction set encompasses data, parallelism (SPL), movement, arithmetic, control flow, and various addressing modes. Warrior generation and mutation are governed by the GPT-4.1-mini LLM, prompted with VM semantics, Redcode grammar, opcode/modifier/addressing documentation, and a warrior example.

• Novel warrior generation: Prompted with “Write a novel Redcode warrior…”.
• Mutation: Provided with parent code plus “Modify this warrior to improve its performance against these opponents…”.

The LLM operates in zero-shot mode with fixed prompt and does not utilize any fine-tuning. This architecture enables both creative synthesis and guided optimization without explicit domain adaptation.

4. Experimental Setup and Metrics

DRQ experiments are conducted on an 8,000-cell Core War VM, with a battle horizon of $\mathcal T = 80,000$ timesteps and 20 random seeds per matchup. Warriors are separated by at least 100 addresses upon initialization. Code length is restricted to 100 lines, supporting up to 8,000 threads.

Benchmarks:

• Static baseline: 294 human-designed warriors
• Generality: 317 held-out warriors

Quantitative metrics include:

Metric	Definition	Use Case
Generality	Fraction of held-out warriors defeated/tied by $w$	Measures cross-domain robustness
Phenotype $\phi(w)$	$\mathbb{R}^{317}$ fitness vector vs held-out warriors	Diversity and convergence analyses
Genotype $g(w)$	Embedding via OpenAI’s text-embedding-3-small/large	Code similarity, functional equivalence
Diversity	Variance of $\phi(w)$ across independent DRQ runs	Measures convergent evolution pressure
Convergence	$\|\phi_t - \phi_{t-1}\|$ (phenotype change per round)	Tracks landscape drift and fixpoints
Cycles	Triplets $(a,b,c)$ with nontransitive defeat relations	Detects cyclic dominance phenomena

5. Empirical Results and Dynamical Phenomena

DRQ exhibits several distinctive evolutionary outcomes:

• Static vs Red Queen: Zero-shot LLM defeats 1.7% of human warriors; best-of-8 LLM sampling covers 22.1%. Single-round evolution against each human yields specialists collectively defeating/tieing 96.3% of humans, but individual evolved warriors only defeat 27.9% on average, indicating severe overfitting.
• Red Queen runs (96 seeds, varied $K$): Generality increases over rounds ($p < 10^{-6}$); phenotypic variance decreases (convergent evolution); rate of phenotypic change slows (landscape drift slows); genotypic variance in embedding space remains constant (many encodings per phenotype).
• Cycles and diversity: Small $K$ yields frequent cyclical dominance; large $K$ (full history) reduces cycles by 77% (hall of fame stabilization). MAP-Elites is essential for late-round performance; its removal (single-cell archive) is detrimental.
• Behavioral archetypes: MAP-Elites heatmaps show top warriors combine high parallelism (numerous SPL threads) and broad memory access. Typical elite programs fuse bomber and replicator tactics.
• Generality prediction: Linear regression $g(w)\rightarrow$ generality achieves test $R^2=0.46$ with large embedding, indicating partial LLM domain understanding and feasibility of cheap surrogate models.

6. Broader Implications and Limitations

The DRQ framework demonstrates a minimal yet effective instantiation of adversarial adaptation, providing a sandbox for continual arms races that yield robustness and convergent phenotypes. Core War’s Turing-complete VM supports safe experimentation in offensive–defensive coevolution, relevant to cybersecurity.

Potential extensions include:

• Transfer to real networks: The LLM + quality/diversity self-play loop may generalize to exploit and defense discovery in complex adversarial environments.
• Application to drug resistance: Analogous continual adaptation could inform evolutionary strategies in biomedical domains.
• Open-ended multi-agent domains: DRQ offers a template for scalable adversarial coevolution.

Notable limitations:

• Computational expense: Battle simulations are resource-intensive (up to $80,000$ steps × thousands of warriors).
• Evolutionary scope: DRQ tracks a single lineage; it lacks explicit ecosystem coevolution (no recombination, parallel populations).
• Strategy optimization: No explicit equilibrium or meta-strategy calculation (cf. PSRO).
• Convergence rate: Phenotype convergence is slow (exponential rounds for full fixpoint).

A plausible implication is that dynamic Red Queen objectives, implemented via self-play and LLM-driven mutation, may yield more general and robust adversarial programs than static optimization. DRQ establishes a tractable and extensible research agenda for evaluating LLM-guided evolution and adversarial robustness in artificial life, cybersecurity, and related multi-agent domains (Kumar et al., 6 Jan 2026).