Introspective Online EM (IOEM)

• The paper introduces IOEM, an online EM framework using introspective step-size adaptation and divergence-based inertia to enhance convergence for latent-variable models.
• It offers closed-form or pseudo-batch updates for exponential-family models, ensuring reliable performance under standard stochastic approximation conditions.
• IOEM efficiently summarizes streaming data with sufficient statistics, supporting applications like vision–language adaptation and time-series modeling while maintaining storage efficiency.

Introspective Online Expectation Maximization (IOEM) encompasses a family of algorithms for performing expectation-maximization (EM) in an online setting, where updates are computed sequentially as new data arrives. IOEM is designed for latent-variable models in contexts where data is observed in a streaming fashion or where storage and batch processing are infeasible. Its defining features include introspective mechanisms for step-size adaptation, divergence-based inertia for regularization, and specialized strategies for introspection in specific tasks, such as vision–LLM adaptation and sequential Monte Carlo estimation.

1. Theoretical Formulation and Objectives

IOEM generalizes the classical EM algorithm to an online or streaming context by incorporating two principal modifications: replacing the batch E- and M-steps with recursive or sequential analogues and introducing introspective adaptations such as step-size tuning or sample weighting based on uncertainty. For latent variable models where the complete-data likelihood takes an exponential family form, IOEM provides closed-form or pseudo-batch online parameter updates that are provably convergent under standard step-size regimes.

The divergence-based interpretation of IOEM introduces an “inertia” term to the M-step objective, enforcing proximity between the updated parameter $\theta^{(t+1)}$ and the previous estimate $\theta^{(t)}$ by penalizing their KL-divergence. Given a (mini)batch $V^{(t)}$ at iteration $t$:

$$Q_{\text{online}}(\theta\mid\theta^{(t)}) = \mathbb{E}_{Z\mid V^{(t)},\,\theta^{(t)}}\left[\log p(V^{(t)},Z;\theta)\right] - \frac{1}{\eta^{(t)}} D_{\text{KL}}\left(p(\cdot ; \theta^{(t)}) \, \| \, p(\cdot ; \theta)\right)$$

(Amid et al., 2019).

This yields online updates as weighted averages of sufficient statistics, with weights governed by a decaying $\eta^{(t)}$. The approach unifies the “observation-level” and “model-level” views of EM, treating the online objective as the sum of divergences from singleton models (recent observations) plus an inertia regularizer.

2. Algorithmic Instantiations

IOEM presents several practical instantiations across model types and domains, including mixture models, hidden Markov models, Kalman filters, vision–LLM adaptation, and sequential Monte Carlo EM.

2.1 Exponential-family Mixture Models

For a mixture of exponential-family components, the online M-step with inertia adopts the following structure:

$$\pi_h^{(t+1)} = \frac{\frac{1}{\eta}\,\pi_h^{(t)} + \frac{1}{|V^{(t)}|} \sum_{n} \gamma_{n,h}}{\frac{1}{\eta} + 1}, \qquad \eta_h^{(t+1)} = g^{-1}\left( \frac{\frac{1}{\eta}\,\pi_h^{(t)} g(\eta_h^{(t)}) + \frac{1}{|V^{(t)}|} \sum_{n} \gamma_{n,h} \phi(v_n)}{\frac{1}{\eta} \pi_h^{(t)} + \frac{1}{|V^{(t)}|} \sum_{n} \gamma_{n,h}} \right)$$

(Amid et al., 2019).

2.2 Vision–LLM Test-Time Adaptation

In FreeTTA, a variant tailored for adapting vision–LLMs (VLMs) such as CLIP at test time, IOEM is instantiated as follows (Dai et al., 9 Jul 2025):

• The latent space of VLM features is modeled by a Gaussian Mixture Model (GMM), one component per semantic class.
• Each incoming sample is processed sequentially: its GMM posteriors are computed (E-step), then the mixture parameters (means, priors, shared covariance) are updated via a weighted running average (M-step).
• The per-sample update is weighted by the self-entropy of the base VLM’s zero-shot predictions, with sample weight $w_t = \exp(-\beta\,H(x_t))$, where $H(x_t)$ is the entropy of CLIP outputs.
• At inference, predictions interpolate the base zero-shot CLIP logit and the GMM generative logit: $$\ell^{\text{final}}_y(x_t) = \ell^{\text{CLIP}}_y(x_t) + \alpha \ell^{\text{GMM}}_y(x_t)$$

2.3 Adaptive Step-Size Regression

In latent-variable time-series models, such as state-space or stochastic volatility models, an alternate IOEM approach adaptively determines the online learning rate $\gamma_t$ via regression on estimated or pseudo-independent parameter updates:

$$\gamma^{\text{reg}}_{t+1} = (|\hat\beta_1| + \hat\sigma_1) / \hat\sigma_0$$

where $\hat\beta_1$ and $\hat\sigma_1$ are the slope and its standard error from a weighted linear regression of pseudo-independent parameter increments, and $\hat\sigma_0$ is the intercept’s standard error. The learning rate is then capped within $[(t+1)^{-1}, (t+1)^{-c_{\min}}]$ for $c_{\min}>1/2$ (Henderson et al., 2018).

3. Storage Efficiency and Sufficient Statistics

IOEM algorithms are designed to be storage-free in the sense of not retaining raw data or entire histories. Instead, all past information is summarized via streaming updates of sufficient statistics such as soft counts, class means, covariances, or aggregated sufficient statistics for each model parameter. For instance, in FreeTTA (Dai et al., 9 Jul 2025), only the GMM’s soft counts ($N_y$), means ($\mu_y$), shared covariance ($\Sigma$), and total effective count ($n$) are required, with no storage or revisit of individual test samples.

This suffices for both introspective adaptation—where mixture parameter evolution encodes intrinsic global structure—and computational efficiency, with time and space per update linear in model and statistic cardinalities.

4. Convergence Analysis

Under standard regularity conditions (e.g., exponential-family complete-data likelihood, compact or suitably restricted parameter space, bounded step-size sequences), IOEM converges almost surely to stationary points of the expected log-likelihood function (Amid et al., 2019, Henderson et al., 2018). Convergence follows from two properties:

• The online objective, with inertia, preserves monotonicity and lower-boundedness.
• The learning rate sequence $\{\eta^{(t)}\}$ or $\{\gamma_t\}$ is chosen so that $\sum_t \eta^{(t)} = \infty$ and $\sum_t (\eta^{(t)})^2 < \infty$, mirroring stochastic approximation requirements.

When introspective, regression-based learning rates are used, they are explicitly capped so that the required series divergence and squared summability conditions hold (i.e., in the Robbins–Monro regime) (Henderson et al., 2018).

5. Empirical Evaluation

Experiments with IOEM have been conducted across multiple domains:

5.1 Vision–Language Test-Time Adaptation

On cross-domain image recognition and out-of-distribution ImageNet variants, FreeTTA employing IOEM produces significant gains over zero-shot CLIP and other state-of-the-art TTA methods. For instance, on CLIP-ViT-B/16:

• Cross-domain: top-1 accuracy increases from 64.59% (baseline) to 68.42% (+3.83 points) (Dai et al., 9 Jul 2025).
• OOD ImageNet: average accuracy rises from 59.42% to 64.42% (+5.00 points), outperforming prior methods by 1.6–3.9 points.

Ablation confirms the necessity of online mean/covariance updates and VLM-based weighting.

5.2 Latent Variable Time Series

In stochastic volatility and autoregressive models, introspective regression-based IOEM matches or exceeds optimally-tuned OEM/BEM learning rates in both accuracy and variance after $10^5$ updates, particularly in scenarios where convergence rates differ substantially between parameters (Henderson et al., 2018).

5.3 Synthetic and Real-World Mixture Models

Divergence-based IOEM has been validated empirically on synthetic datasets for mixtures, Kalman filters, and HMMs, exhibiting stable monotonic likelihood ascent and correct distributed model merging (Amid et al., 2019).

6. Distributed and Modular Model Fusion

The relative-entropy sum framework underlying IOEM enables principled merging of estimates from multiple distributed workers. For hidden-variable models, combining $M$ local estimates $\{\theta^{(m)}\}$ reduces to minimizing a weighted sum of KL-divergences to a global parameter:

$$\theta^{(\text{comb})} = \arg\min_\theta \sum_{m=1}^M \alpha_m\, D_{\text{KL}}(p(\cdot; \theta^{(m)}) \| p(\cdot; \theta))$$

This convex combination applies to the complete-data sufficient statistics, generalizing IOEM to ensemble, parallel, and federated contexts (Amid et al., 2019).

7. Applications and Implementation Notes

IOEM is particularly advantageous in settings with streaming data, constrained storage, or parameter heterogeneity. Example applications include test-time domain adaptation for VLMs, distributed learning, and online estimation in large-scale hidden Markov models or dynamical systems.

Practical guidance includes:

• Adopting per-parameter regression for introspective step-size control in high-dimensional models (Henderson et al., 2018).
• Using hand-crafted semantic prototypes and entropy weighting for robust online adaptation in vision-deep learning (Dai et al., 9 Jul 2025).
• Leveraging pseudo-batch simulations to approximate inertia terms in non-closed-form cases (Amid et al., 2019).

Parameter settings such as step-size caps, entropy weighting, and combination interpolation coefficients are fixed or decayed as required for stability and convergence. Python implementations for certain cases (e.g., SMC-EM) are available (Henderson et al., 2018).

---

In summary, Introspective Online EM unifies online EM, adaptive learning rates, divergence-based regularization, and storage-efficient streaming updates for latent-variable models. Its introspective mechanisms—including uncertainty-weighted updating, regression-based step-size control, and sufficient-statistic compression—enable robust, training-free adaptation and estimation under challenging distributional and infrastructural constraints (Dai et al., 9 Jul 2025, Amid et al., 2019, Henderson et al., 2018).