Visual Information Fidelity (VIF) Measure

• Visual Information Fidelity (VIF) is an information-theoretic index that measures the proportion of visual information retained after image distortion.
• It employs wavelet decompositions and probabilistic models, including GSM and MGGD, to accurately capture both standard and heavy-tailed distortions.
• The GGSM-VIF extension adaptively estimates local parameters, enhancing sensitivity and performance for assessing user-generated content distortions.

Visual Information Fidelity (VIF) is an information-theoretic full-reference image quality assessment (IQA) index designed to quantify the visual similarity between a reference image and its distortion, grounded in probabilistic modeling of natural scene statistics (NSS) and incorporating models of the human visual system’s (HVS) information processing. The VIF index originally relies on a Gaussian Scale Mixture (GSM) model of natural image wavelet subband coefficients and has recently been generalized to employ Multivariate Generalized Gaussian Distributions (MGGD), enabling improved robustness to atypical or severe image distortions and better modeling of empirical coefficient distributions, as seen in user-generated content (Venkataramanan et al., 2023).

1. Foundational Principles of the VIF Measure

The VIF metric is founded on the premise that visual quality can be quantified via information fidelity: the proportion of visual information preserved between a reference image and its distortion, as measured by mutual information rates in the domain of natural image statistics. The computation involves the following key components:

• Wavelet or Steerable-Pyramid Decomposition: The image is partitioned into $K$ subbands via a multi-scale, multi-orientation transform.
• Local Coefficient Modeling: Within each subband $k$, an $M$-dimensional vector of coefficients $C_i^k$ (reference) is extracted from the $i$th spatial neighborhood.
• Distortion Model: The corresponding distorted coefficients $D_i^k$ are assumed to follow $D_i^k = g_i^k C_i^k + V_i^k$, where $g_i^k$ is a deterministic gain, $V_i^k \sim N(0, \sigma_v^2 I)$ is additive Gaussian noise.
• Observer Model: To approximate perceptual mechanisms, the observed coefficients include “neural noise” $N_i^k, N_i^{\prime k} \sim N(0, \sigma_n^2 I)$, yielding $E_i^k = C_i^k + N_i^k$ (reference) and $F_i^k = g_i^k C_i^k + N_i^{\prime k}$ (distorted).

The latent coefficients $C_i^k$ are crucially modeled using a scale mixture, capturing heavy-tailed marginal statistics empirically observed in natural images.

2. Gaussian Scale Mixture (GSM) Model and Original VIF Definition

The original VIF assumes a GSM model for wavelet coefficients:

• $C = Z U$, where $U \sim N(0, \Sigma_u)$ and the mixing variable $Z \geq 0$ is independent and positive.
• This captures local variance and heavy tails using the underlying Gaussian vector $U$ and spatially-varying scale $Z$.

Mutual information rates, conditioned on a fixed realization $Z = z$, are derived as:

• Reference: $$I(C_i^k; E_i^k | Z_i^k = z) = h(C_i^k + N_i^k | Z_i^k = z) - h(N_i^k)$$
• Distorted: $$I(C_i^k; F_i^k | Z_i^k = z) = h(g_i^k C_i^k + N_i^{\prime k} | Z_i^k = z) - h(N_i^{\prime k})$$

Summing over all subbands and spatial neighborhoods yields total “source” and “distorted” information:

• $$I_\text{source} = \sum_{k=1}^{K} \sum_{i=1}^{N} I(C_i^k; E_i^k | Z_i^k)$$
• $$I_\text{dist} = \sum_{k=1}^{K} \sum_{i=1}^{N} I(C_i^k; F_i^k | Z_i^k)$$

The VIF index is defined as: $$\mathrm{VIF} = \frac{I_\text{dist}}{I_\text{source}}$$

This ratio reflects the relative amount of visual information that survives distortion.

3. Generalized GSM (GGSM) and the Multivariate Generalized Gaussian Distribution (MGGD)

Empirical image data—particularly user-generated content—often exhibit deviations from the Gaussian assumption. The MGGD provides a more flexible modeling framework, with probability density

$$f_U(u) = \frac{\beta}{2^{n/2} \pi^{n/2} |\Sigma|^{1/2} \Gamma(n/(2\beta))} \exp\left[ - (u^{T}\Sigma^{-1}u)^{\beta/2} \right]$$

where $n$ is the coefficient vector dimension, $\Sigma$ is the scatter matrix, and $\beta > 0$ is a shape parameter controlling tail-heaviness:

• $\beta = 1$: Gaussian distribution (lightest tails)
• $\beta < 1$: Leptokurtic (heavier tails)
• $\beta > 1$: Sub-Gaussian (lighter tails).

Statistical properties relevant for information-theoretic computations include:

• Covariance: $\mathrm{Cov}(U) = m_2(\beta, n) \Sigma$, with $$m_2(\beta, n) = \Gamma((n+2)/2\beta) / \Gamma(n/2\beta)$$.
• Differential entropy: $$h(U) = \frac{n}{2\beta} -\log \left[ \beta/(2^{n/2} \pi^{n/2} \Gamma(n/2\beta) ) \right] + \frac{1}{2} \log|\Sigma|$$.
• Sample kurtosis, used for $\beta$ estimation, relates to fourth-order moments.

This model underpins the GGSM-VIF extension, wherein the shape parameter $\beta$ and scatter matrix $\Sigma$ are adaptively estimated for each block.

4. Derivation and Mathematical Formulation of VIF Under GGSM

In the GGSM-VIF framework, $C | Z=z \sim z U$ for $U$ as a zero-mean MGGD with shape parameter $\beta$ and scatter $z^2 \Sigma_u$.

The mutual information for each neighborhood, conditioned on $Z=z$, becomes:

• Reference: $$I_\text{GGSM}(C; E | Z=z) = h_\text{MGGD}(0, \beta_s, \Sigma_E) - h_\text{Gauss}(\sigma_n^2)$$, with $\Sigma_E = z^2 \Sigma_u + \sigma_n^2 I$, and $\beta_s$ determined from the empirical distribution of $E$.
• Distorted: $$I_\text{GGSM}(C; F | Z=z) = h_\text{MGGD}(0, \beta_d, \Sigma_F) - h_\text{Gauss}(\sigma_n^2 + \sigma_v^2)$$, with $$\Sigma_F = g^2 z^2 \Sigma_u + (\sigma_n^2 + \sigma_v^2) I$$.

The auxiliary function

$$\psi(\Sigma, \beta, \sigma^2) \equiv h_\text{MGGD}(0, \beta, \Sigma+\sigma^2 I) - h_\text{Gauss}(\sigma^2)$$

parameterizes the contribution per block and subband. Summing these across all neighborhoods and subbands, the generalized VIF reads: $$\mathrm{VIF}_\text{GGSM} = \frac{\sum_{k, i} \psi(g_{i}^k{}^{2} z_{i}^k{}^{2}\Sigma_u^k, \beta_{d}^k, \sigma_n^2 + \sigma_v^2)}{\sum_{k, i} \psi(z_{i}^k{}^{2}\Sigma_u^k, \beta_{s}^k, \sigma_n^2)}$$ A plausible implication is that this flexible adaptation to local tail behavior enables more accurate reflection of perceptually impactful distortions, especially in non-Gaussian or heavy-tailed regimes (Venkataramanan et al., 2023).

5. Estimation of MGGD Parameters in Practice

For each subband and spatial block, the estimation of MGGD parameters $(\Sigma, \beta)$ is performed as follows:

• Compute the sample covariance $\hat{C} = \frac{1}{N}\sum x_i x_i^T$ and sample Mardia's kurtosis $\hat{\gamma}_2$.
• Solve, via root-finding, for $\beta$ from the theoretical MGGD kurtosis formula: $$\gamma_2 = n(n+2)\left[\frac{\Gamma((n+4)/2\beta)\Gamma(n/2\beta)}{\Gamma((n+2)/2\beta)^2} - 1\right]$$ by setting $\hat{\gamma}_2 = \gamma_2$.
• Set $\Sigma = \hat{C} / m_2(\beta, n)$ with $m_2$ as above.
• Repeat for “noisy” observed blocks to obtain $\beta_d$, $\Sigma_E$, $\Sigma_F$ as required by the model.

Empirically, this estimation scheme allows $\beta$ to vary by subband and block, providing enhanced modeling capacity over the fixed-Gaussian assumption.

6. Comparison: GSM-VIF Versus GGSM-VIF

Aspect	GSM-VIF	GGSM-VIF
Tail Modeling	$\beta=1$ (pure Gaussian, fixed)	$\beta$ adaptively estimated, subband- and noise-specific
Distortion Handling	Sensitive mainly to Gaussian-like noise	Responsive to complex, heavy-tailed distortions
Empirical Performance	Noted limitations on UGC	Gains of 2–5 points in Spearman rank on UGC (prelim.)

The GGSM-VIF generalization enhances sensitivity to local distributional changes, particularly in challenging user-generated content. This increased fidelity is attributed to its adaptive modeling of local kurtosis and tail behavior that are not captured under the original GSM model (Venkataramanan et al., 2023). Theoretically, GGSM-VIF is expected to yield improved discrimination of distortion-induced structure changes.

7. Implementation and Application Considerations

To compute the VIF (in its original or generalized form), the following workflow is performed:

• Decompose both reference and distorted images using a wavelet or steerable-pyramid, organizing subband data.
• For each neighborhood, estimate parameters $\Sigma$, $\beta$ for both reference and observed/distorted sets.
• Form the relevant $\psi$-function for each block and aggregate information across blocks and subbands as dictated by the formal equations.
• Compute the VIF (or VIF$_\text{GGSM}$) index as the ratio of summed information rates.

The comprehensive mathematical derivation, parameter estimation routines, and auxiliary formulas provide a self-contained framework for implementing both GSM-VIF and GGSM-VIF approaches to full-reference image quality assessment (Venkataramanan et al., 2023).