Computing Evolutionarily Stable Strategies in Imperfect-Information Games

Abstract: We present an algorithm for computing evolutionarily stable strategies (ESSs) in symmetric perfect-recall extensive-form games of imperfect information. Our main algorithm is for two-player games, and we describe how it can be extended to multiplayer games. The algorithm is sound and computes all ESSs in nondegenerate games and a subset of them in degenerate games which contain an infinite continuum of symmetric Nash equilibria. The algorithm is anytime and can be stopped early to find one or more ESSs. We experiment on an imperfect-information cancer signaling game as well as random games to demonstrate scalability.

• The paper presents a novel computational framework that computes evolutionarily stable strategies (ESS) using a sequence-form representation and QCQP verification.
• It iteratively identifies symmetric Nash equilibria and rigorously checks for mutant resistance, addressing issues like degeneracy and scalability in game complexity.
• Experimental results from a cancer signaling game demonstrate rapid computation and reveal that robust pure strategies outperform vulnerable mixed ones.

Computing Evolutionarily Stable Strategies in Imperfect-Information Extensive-Form Games

Background and Motivation

Evolutionarily stable strategy (ESS) is the central equilibrium concept in evolutionary game theory, providing a refinement of Nash equilibrium which is robust to mutant invasions under small population perturbations. While classical ESS analysis focuses on symmetric normal-form games, extensive-form games (EFGs) with imperfect information arise naturally in many biological, signaling, and ecological contexts and present unique challenges for ESS computation. Existing work has characterized evolutionary stability in sequential games with imperfect information from theoretical and dynamical systems perspectives, but there has been no practical computational framework for identifying ESSs in general extensive-form settings. This paper addresses that gap and presents an algorithm for computing all evolutionarily stable strategies in symmetric perfect-recall extensive-form games, extending to multiplayer scenarios.

Formalisms and Definitions

The paper rigorously formalizes the ESS concept for imperfect-information EFGs via a sequence-form representation, which efficiently encodes the exponential pure strategy space in terms of information-set-consistent action-sequences. For two-player symmetric EFGs, the utility and information constraints guarantee symmetry and feasibility of behavioral realization plans. The classical ESS definition is extended: a behavioral strategy $\boldsymbol{\sigma}^\star$ is ESS if for any strategy $\boldsymbol{\sigma} \neq \boldsymbol{\sigma}^\star$, either $\boldsymbol{\sigma}^\star$ strictly outperforms the mutant $\boldsymbol{\sigma}$ when both are played against the incumbent, or in the case of a tie, $\boldsymbol{\sigma}^\star$ performs strictly better than $\boldsymbol{\sigma}$ in head-to-head competition. This notion is implemented directly in the sequence form, with constraints ensuring both feasibility and separation from label degeneracy.

Algorithmic Framework

The proposed algorithm is an anytime procedure, iteratively identifying symmetric Nash equilibria (SNEs) and rigorously verifying their evolutionary stability via a nonconvex quadratic programming check for profitable mutants.

Key steps are as follows:

• SNE Identification: Using a quadratically-constrained feasibility program in sequence form, the method finds initial and subsequent SNEs, maximizing minimal L2 separation to avoid duplicate enumeration.
• ESS Verification: For each candidate SNE, a quadratically-constrained quadratic program (QCQP) searches the space of feasible mutants (behavioral strategies distinct from the incumbent) for potential violations of ESS conditions. If no such mutant exists even under numerical perturbation constraints, the strategy is labeled as ESS.
• Degeneracy Handling and Scalability: The algorithm is robust to degeneracy—the existence of a continuum of SNEs—by halting on numerical exhaustion or upon user-defined criteria, and it can efficiently process moderate-sized games through the use of global quadratic optimization.
• Extension to Multiplayer Games: The methodology generalizes conceptually to $n$-player symmetric EFGs, with noted exponential scaling in auxiliary variables but direct applicability to small multiplayer domains.

Experimental Results

The paper validates the framework on a biologically motivated cancer signaling game, simulating clonal responses under uncertain therapy with private noisy signals and phenotypic decision nodes. The extensive-form tree for this game encapsulates chance moves for therapy application, information-set partitioning for signals, and subsequent phenotype selection.

Figure 1: Extensive-form tree structure of the cancer signaling game; Nature chooses therapy intensity, players receive private signals, and select phenotypes at respective private information sets.

Application of the algorithm produces the complete set of SNEs and ESSs for the game. Strong numerical results:

• Numerical Feature: The cancer signaling game yields seven SNEs, three of which are ESSs. All ESSs correspond to behavioral strategies which play a pure phenotype in response to an unfavorable private signal (either $P$, $R$, or $Q$), with nonpure mixtures vulnerable to mutant invasion due to payoffs equalization.
• Computation Time: The full equilibrium and stability analysis completes in $0.287$ seconds on standard hardware.

Further experiments on randomly generated symmetric EFGs with variable numbers of signals ($S$) and actions ($A$) demonstrate:

• Scalability: Median computation times remain sub-second for small ($A \leq 4$) games, and typically within seconds up to $A=6$, $S=5$.
• Structural Trends: The number of SNEs grows rapidly with game complexity (signals/actions), while ESSs remain much rarer, emphasizing the selectivity of evolutionary stability in sequential settings.

Implications and Prospects

The formalization and algorithmic treatment of ESS in extensive-form imperfect-information games opens important avenues in the modeling of biological populations, evolutionary signaling, tumor ecology, and adaptive multi-agent systems. The ability to enumerate all ESSs allows for direct comparison of evolutionary robustness across alternative strategic programs, guiding the identification of stable behavioral schemes in settings where information is partial and actions are sequential. It also enables fine-grained modeling of resistance and invasion dynamics, as illustrated in the cancer signaling scenario.

On the theoretical front, the algorithm uncovers the combinatorial structure underlying ESS rarity in high-dimensional sequential games, a phenomenon that may inform future mathematical classification of stability conditions beyond classical normal form analysis. Practically, the method can underpin evolutionary modeling in domains such as tumor–immune interactions, multi-clone ecological conflict, and adaptive treatment regimes, potentially integrating stochastic game-theoretic approaches with empirical genomic or phenotypic data. Although computational hardness constrains scalability for large multiplayer settings, improvements in global optimization solvers and tailored decomposition strategies offer clear future trajectories.

Conclusion

This paper delivers a rigorous computational framework for evolutionarily stable strategy analysis in symmetric EFGs of imperfect information, combining exact sequence-form encoding with robust quadratic verification of mutant resistance. Empirical results indicate practical scalability and strong discriminatory power of the ESS concept over Nash equilibrium in sequential biological and random game models. The methodology provides both practical tools and theoretical foundations for evolutionary game analysis in structured, information-rich environments, with significant implications for the design, prediction, and manipulation of robust strategic behaviors under partial information. Future research will aim to enhance scalability, address degeneracy, extend to multiplayer domains, and integrate with empirical biological and ecological datasets.