Sequential Estimation of Dynamic Discrete Choice Models with Unobserved Heterogeneity

Abstract: Estimating dynamic discrete choice models with unobserved heterogeneity is computationally costly because it requires repeatedly solving fixed-point equations for all unobserved types. We develop the EM-NPL(q) framework that combines the Expectation-Maximization (EM) algorithm with an inner fixed-point solver truncated to q iterations. For the workhorse class of linear-in-parameters models, we establish a truncation-invariance result: for any q$\geq$1, EM-NPL(q) is numerically identical to the EM-NPL estimator that solves the inner fixed-point problem to convergence. Therefore, the choice of q affects computation but not statistical properties. We also establish consistency, asymptotic normality of our estimator, and local convergence of the EM-NPL(q) algorithm. In Monte Carlo simulations, EM-NPL(q) reduces runtime by at least 20% and can be 3--5 times faster. In an application to cola demand, we show that ignoring unobserved heterogeneity understates long-run own-price elasticities by up to 60%, short-run elasticities by up to 85%, and compensating variation from a soda tax by up to 90%.

• The paper introduces the EM-NPL(q) framework, which truncates inner fixed-point iterations without sacrificing statistical accuracy in DDC models.
• It demonstrates truncation invariance for linear-in-parameters models, ensuring that computational shortcuts do not bias inference.
• Simulation and empirical results highlight significant computational gains and improved estimation of consumer price elasticities in high-dimensional settings.

Sequential Estimation of Dynamic Discrete Choice Models with Unobserved Heterogeneity via EM-NPL($q$)

Introduction and Motivation

This paper addresses computational and inferential challenges in the estimation of dynamic discrete choice (DDC) models with finite mixture (latent class) unobserved heterogeneity. Incorporating unobserved heterogeneity in forward-looking structural models is empirically central but computationally demanding, especially for large state spaces and multiple types. Estimation typically requires solving numerous fixed-point equations (e.g., associated with value functions or conditional choice probabilities, CCPs) for each latent type at every parameter evaluation, making existing protocols for DDC models impractical on large-scale empirical applications.

The core contribution is the EM-NPL($q$) framework, which intertwines the Expectation-Maximization (EM) algorithm in a finite mixture context with a pseudo-likelihood approach based on sequential estimation of structural DDC models. Critically, the paper proposes running only $q$ iterations of the inner fixed-point algorithm (for approximating value functions or CCPs) at each EM step, rather than solving the fixed point to tight convergence. For the principal class of linear-in-parameters models, the authors establish truncation invariance: statistical properties and numerical solutions are unaffected by the value of $q \geq 1$—i.e., computational choices are decoupled from inferential outcomes.

Model Structure

The framework allows for general single-agent and dynamic game DDC models with finite mixture unobserved heterogeneity, as in [kasahara2009nonparametric, arcidiacono2011conditional]. Each market/agent’s type is randomly drawn and fixed over time, generating mixture weights $\pi_m$ over $M$ possible types. At each period, conditional on latent type $m$ and observed state vector $x_t$, agents choose among discrete actions $a_t\in A$ to maximize discounted expected utility. Model structure is standard: additive separability of utility, conditional independence, finite state and action spaces, and (optionally) the Markov Perfect Equilibrium concept in games.

Utility functions linear in parameters are highlighted, which enables operator-theoretic simplifications in the policy valuation, Euler equation, and efficient pseudo-likelihood frameworks.

The EM-NPL($q$) Framework

The EM-NPL($q$0) algorithm consists of the standard EM updates on type probabilities and mixing weights (E-step), and maximum likelihood/pseudo-likelihood parameter updates for each type (M-step). Each M-step requires a computation of nuisance parameters (e.g., value/shadow functions $q$1 or CCPs $q$2) satisfying a type-specific fixed-point equation:

$q$3

where $q$4 is the mapping induced by the DDC model (e.g., policy evaluation, Bellman, or Euler operator). Rather than solving this equation to fixed-point convergence, the EM-NPL($q$5) approach only applies $q$6 steps of a chosen iterative inner algorithm (successive approximation, GMRES, Newton-type), denoted as $q$7.

Estimation is performed by recursively updating parameters and mixture probabilities, using truncated inner solutions, until overall convergence in $q$8, $q$9, and $q$0.

Theoretical Properties

Truncation Invariance

For linear-in-parameters settings, a key result is that the mapping $q$1 is independent of the current parameter candidate $q$2 (“$q$3-separability”), so the nuisance solution depends only on previous values and not on current $q$4. Therefore, as long as the operator has a unique fixed point, the EM-NPL($q$5) estimator is identical to the full-solution EM-NPL estimator for all $q$6: inner truncation (the value of $q$7) affects computation, but never the numerical or inferential properties of the outer estimator.

This decoupling is significant: it avoids the “fixed-point bias” and provides a theoretical guarantee that outer layer estimation and inference can proceed as if the inner layer were solved exactly.

Asymptotic Properties

Under standard regularity and identification conditions, the estimator is consistent and asymptotically normal. The influence matrix and asymptotic covariance reflect the outer likelihood geometry, with corrections only necessary when the fixed-point operator $q$8 depends on the current parameter (as in nonlinear utility models or the EPL estimator). For linear-in-parameters models, standard errors are unaffected by $q$9. For nonlinear cases, variance errors are $q \geq 1$0, matching the rate of contraction of the inner solver.

Local Convergence

Local convergence of the EM-NPL($q \geq 1$1) algorithm is analyzed via the spectral radius of a block convergence matrix $q \geq 1$2. Provided the exact EM-NPL algorithm converges locally, a sufficiently large $q \geq 1$3 ensures local convergence for the truncated variant. The analysis delivers explicit bounds on the required number of outer iterations as a function of $q \geq 1$4, showing exponential improvements for superlinear inner algorithms (GMRES, Newton-type), and quantifying the dependence on the spectral properties of $q \geq 1$5.

Practical Guidance

The paper provides concrete implementation recommendations, emphasizing:

1. Benchmarking Inner Solvers: On the initial fixed-point problem, one should empirically screen successive approximation, GMRES, Newton, and Anderson acceleration schemes and select the fastest (often GMRES).
2. Fixed-Point Equation Selection: For linear-in-parameters models, the policy valuation or EPL (efficient pseudo-likelihood) fixed-point is preferable, both because of computational tractability and statistical efficiency; for models with finite dependence, Euler equations offer rapid contraction.
3. Choice of $q \geq 1$6: For GMRES or Newton, $q \geq 1$7–$q \geq 1$8 suffices; for successive approximation, larger $q \geq 1$9 may be needed, particularly for high discount factors. For linear-in-parameters settings, $\pi_m$0 can be chosen based purely on computation time.

Monte Carlo Simulations

Extensive simulations are conducted in both single-agent and dynamic game contexts. The principal findings include:

• Truncation Invariance in Practice: For the policy-valuation (GMRES and successive approximation) and Euler equation methods, MSE is empirically invariant with $\pi_m$1, reconfirming the truncation-invariance theorem.
• Computational Efficiency: PV_GMRES dominates all alternatives in computational efficiency. For example, for $\pi_m$2, PV_GMRES($\pi_m$3) achieves 3x–5x speedup relative to PV_SA and Newton.
• Statistical Efficiency: The EPL-based estimator achieves lower MSE but is one order of magnitude slower than policy-valuation-based NPL. EM-NPL($\pi_m$4) algorithms with truncated inner loops reduce EPL runtime by roughly 40%.
• Dynamic Games: In games, the EPL estimator is statistically superior but has higher computational costs; spectrum-based CCP updates enhance convergence.

Empirical Application: Demand for Cola

The method is applied to household-level demand for cola products using a large scanner panel dataset with a $\pi_m$5-point state space, across up to $\pi_m$6 latent types and three income groups. The computational time per estimation is approximately linear in $\pi_m$7 (see Figure 1).

Figure 1: Computational time of EM-NPL($\pi_m$8) by number of latent types $\pi_m$9.

The main empirical findings are:

• Elasticity Understatement: Ignoring unobserved heterogeneity (i.e., mixture types) can understate long-run own-price elasticities by up to 60%, short-run elasticities by up to 85%, and the compensating variation of a soda tax by up to 90%.
• Heterogeneity in Elasticities: Unobserved types exhibit substantial heterogeneity in price sensitivity, with one or two types bearing almost all the welfare cost of soda taxes (see Figures 2–5).

Figure 2: Long-run own-price elasticities for low income group as a function of $M$0.

Figure 3: Short-run own-price elasticities for low income group as a function of $M$1.

Figure 4: Compensating variation (CV) by number of latent types $M$2 for low income households; right panel shows CV by type at $M$3.

Figure 5: Compensating variation by number of children for low income group.

This underscores the substantive empirical bias arising from ignoring heterogeneity and validates the practical and computational feasibility of the proposed approach in high-dimensional empirical settings.

Implications and Future Directions

The EM-NPL($M$4) framework enables scalable, statistically robust estimation of complex DDC models with unobserved heterogeneity. By decoupling computational and inferential concerns, researchers can implement large-scale latent-class DDC models in complex games and panel datasets without loss of statistical efficiency. The practicality of the framework is demonstrated empirically and in demanding Monte Carlo simulations.

The methodology is directly extensible to more general forms of unobserved heterogeneity, broader classes of DDC models (dynamic oligopoly, multi-agent interaction games), and recent developments in efficient pseudo-likelihood techniques. Further directions include integration with doubly robust or Neyman-orthogonal estimation protocols, exploration of nonparametric mixture identification, and harnessing distributed or parallel inner solvers in massive state spaces.

Conclusion

This paper provides a unified, modular, and computationally efficient framework for the (pseudo)maximum likelihood estimation of DDC models with finite mixture unobserved heterogeneity. Truncating the inner fixed-point solver (EM-NPL($M$5)) is proven to be neutral for statistical properties in linear-in-parameters cases, and simulation and application evidence indicates substantial computational gains without sacrificing statistical efficiency. The strategy enables researchers to flexibly adopt powerful, type-rich structural models in empirical work and counterfactual analysis, bridging the gap between econometric theory and modern large-scale computational practice.