Primal-Dual Formulation in Mean Field Games

• Primal-dual formulation is a measure-theoretic framework that couples a primal optimization with a dual maximization, ensuring value matching via occupation measures.
• It converts continuous-time control and mean field games into linear programs by leveraging stochastic process constraints and Hamilton-Jacobi-Bellman (HJB) subsolutions.
• Strong duality under classical regularity regimes enables precise characterization of Nash equilibria even in nonconvex and nonunique solution settings.

A primal-dual formulation is a measure-theoretic or variational framework that simultaneously expresses an optimization problem (the "primal") and a concave or maximization problem (the "dual"), with value-matching and feasibility conditions linking their solutions. In continuous-time control and game-theoretic contexts, primal-dual formulations enable both the characterization of optimal controls/policies and the precise analytical description of all equilibria (e.g., Nash equilibria in continuous-time mean field games). The technical core is often an equivalence between the original closed-loop control problem and a linear program over occupation measures, together with an abstract dual based on subsolutions to a corresponding Hamilton-Jacobi-Bellman (HJB) equation, resulting in strong duality under minimal regularity assumptions (Guo et al., 2 Mar 2025).

1. Measure-Theoretic Primal Formulation for Control and MFGs

Consider a continuous-time controlled diffusion process for a representative agent with state space $\R^d$, action space $A$ (Polish metric), and finite time horizon $[0,T]$. Given a deterministic mean-field flow $\bm\mu = (\mu_t)_{t\in[0,T]} \in P(\R^d)^{[0,T]}$ and initial measure $\rho$, the agent selects a measurable, relaxed Markov policy $\gamma: [0,T]\times\R^d \to P(A)$. The state dynamics follow

$$X_t = X_0 + \int_0^t b^{\bm\mu,\gamma}(s,X_s)ds + \int_0^t \sigma^{\bm\mu,\gamma}(s,X_s) dW_s$$

for appropriate drift/diffusion $b,\sigma$ and $d$-dimensional Brownian motion $W$.

The occupation measure approach replaces stochastic process optimization with a linear program on $$X_+ = M_+(\R^d) \times M_+([0,T]\times\R^d\times A)$$, with $\nu$ representing the law of $X_T$ and $\xi$ encoding time–state–action occupancy. The key linear "martingale-constraint" for smooth test functions $\psi \in W = C_b^{1,2}$ is: $$\int_{\R^d} \psi(T,x) \nu(dx) - \int_{\R^d} \psi(0,x) \rho(dx) = \int_{[0,T]\times\R^d\times A} [\partial_t\psi + {}^{\bm\mu}\psi] \, \xi(dt,dx,da)$$ where ${}^{\bm\mu}\psi$ denotes the controlled generator.

The corresponding primal LP is: $$V_P^{\bm\mu} = \inf_{(\nu,\xi)\in\mathcal{D}_P(\bm\mu)} \left\{ \int f(t,x,a,\mu_t) \xi(dt,dx,da) + \int g(x,\mu_T) \nu(dx) \right\}$$ for bounded measurable running/terminal costs $f,g$.

An equivalence theorem (Thm 3.5) establishes that the value of the occupation-measure LP matches the original closed-loop stochastic control value: $V_{\mathrm{cl}}^{\bm\mu} = V_P^{\bm\mu}$ with a precise correspondence between optimal policies and optimal occupation measures via disintegration and superposition principles.

2. Dual Formulation: HJB Subsolutions and Abstract Duality

Duality is achieved by constructing the adjoint operator $L^*$ acting on smooth test functions $\psi$, leading to dual feasibility conditions:

• Terminal cost upper bound: $g(x,\mu_T) \ge \psi(T,x)$
• Pointwise subsolution constraint: $$\partial_t\psi(t,x) + {}^{\bm\mu}\psi(t,x,a) + f(t,x,a,\mu_t) \ge 0$$

This yields the dual maximization problem over smooth subsolutions of the formal HJB equation: $$V_D^{\bm\mu} = \sup_{\psi \in D_{P^*}(\bm\mu)} \int_{\R^d} \psi(0,x) \rho(dx)$$ where

$$D_{P^*}(\bm\mu) = \{\psi \in C_b^{1,2}([0,T]\times\R^d) : \psi(T,\cdot) \le g(\cdot,\mu_T), \; \partial_t\psi + {}^{\bm\mu}\psi + f \ge 0 \}$$

The formal HJB equation for the value function $V$ takes the shape: $$\partial_t V + \inf_{a \in A} \left\{ {}^{\bm\mu}V(t,x,a) + f(t,x,a,\mu_t) \right\} = 0, \quad V(T,x)=g(x,\mu_T)$$ Weak duality ($V_P^{\bm\mu} \ge V_D^{\bm\mu}$) is immediate by pairing primal-feasible and dual-feasible elements.

3. Strong Duality and Regularity Regimes

Strong duality ($V_P^{\bm\mu} = V_D^{\bm\mu}$) holds when the HJB admits a classical ($C_b^{1,2}$) solution $V$. Under such circumstances, there exists a measurable selector $\phi$ implementing the minimizer in the HJB, and the deterministic feedback policy $\gamma(da|t,x)=\delta_{\phi(t,x)}(da)$ produces a weak solution to the SDE, saturating the dual constraints.

Elliptic/parabolic regularity arguments differentiate two regimes:

• Semilinear HJB (uncontrolled $\sigma$) admit classical solutions via Schauder estimates.
• Fully nonlinear HJB (controlled $\sigma$) with Lipschitz data yield classical solutions via Evans–Krylov theory.

In both, Itô’s formula verifies equality: $V_P^{\bm\mu} = V_D^{\bm\mu} = \int V(0,x) \rho(dx)$ The proof structure involves measurable selection, construction of occupation measures, and demonstration of vanishing duality gap.

4. Primal-Dual Characterization of Nash Equilibria

A Nash equilibrium (NE) for MFGs is a triple $(\bm\mu^*, X^*, \gamma^*)$ such that, given flow $\bm\mu^*$, $(X^*, \gamma^*)$ solves the representative control problem and is consistent, $\mu_t^* = \operatorname{Law}(X_t^*)$.

The primal-dual system characterizing NE (Thm 4.12) is:

• Primal feasibility: $(\xi^*, \mu_T^*) \in \mathcal{D}_P(\bm\mu^*)$
• Dual feasibility: $\psi^* \in D_{P^*}(\bm\mu^*)$
• Value matching: $$\int g(x, \mu_T^*) \mu_T^*(dx) + \int f\,d\xi^* = \int \psi^*(0,x)\rho(dx)$$
• Consistency: time–state marginal of $\xi^*$ is $\mu_t^*\,dt$

When strong duality holds at $\bm\mu^*$, any Nash equilibrium must satisfy this system. Every minimizer or subsolution (whether pure or mixed) is faithfully represented, and joint primal-dual feasibility identifies all NEs.

5. Absence of Convexity and Uniqueness Assumptions

Unlike conventional approaches requiring convexity in the Hamiltonian or uniqueness of the optimizer, this primal-dual framework only presupposes measurability and boundedness of the coefficients $(b, \sigma, f, g)$. It allows for nonconvex, nonunique minimizers, and remains robust even when the HJB equation lacks classical (or continuous) solutions. Subsolution-based dual feasibility permits NE construction on the support of the actual flow, not just globally.

6. Key Technical Ingredients and Proof Structure

Crucial lemmas include:

• Disintegration: every occupation measure $\xi$ can be uniquely decomposed into a Markov kernel $\gamma(da|t,x)$ and a time-marginal $m^X_t(dx)$.
• Superposition principle: continuous solutions to the time-marginal Fokker–Planck equation lift to martingale solutions of the SDE.
• LP Duality theory: abstract weak duality holds whenever primal and dual cones are nonempty.
• PDE estimates (Schauder, Evans–Krylov) guarantee existence of classical HJB solutions in the respective regimes.

These technical ingredients underwrite the full identification of Nash equilibria through matched primal-dual feasibility and value matching, closing both the analytical and measure-theoretic duality gap (Guo et al., 2 Mar 2025).

Table: Analytical Structures in Primal-Dual MFG Formulation

Component	Mathematical Object	Function/Property
Primal LP	Occupation measures $(\nu,\xi)$	Minimizes expected cost given flow
Dual Problem	Smooth subsolutions $\psi$	Maximizes initial value subject to HJB-type inequalities
Constraint	Martingale-constraint Eqn	Enforces valid controlled diffusion paths
NE Characterization	Triple $(\bm\mu^*, \xi^*, \psi^*)$	Feasibility, value matching, law-consistency
Regularity regime	Classical HJB solution	Ensures strong duality, explicit feedback
Extension	Nonconvex/Nonunique regime	Subsolution-based feasibility, generalized NE

This primal-dual analytical framework has established itself as a rigorous and complete characterization tool in continuous-time MFGs, general stochastic control, and beyond, providing full identification of Nash equilibria without restrictive convexity or uniqueness conditions.