Probabilistic Multi-Rater Medical Segmentation

• The paper introduces a dual-latent framework that separates image ambiguity from annotator bias to improve segmentation personalization and diversity.
• It leverages variational inference and statistical-distance losses to achieve state-of-the-art accuracy on benchmarks such as LIDC-IDRI and NPC.
• The approach enables both personalized expert reconstructions and generation of plausible virtual expert outputs for robust clinical applications.

Probabilistic modeling of multi-rater medical image segmentation (ProSeg) refers to a spectrum of methods for incorporating, modeling, and exploiting inter-expert variability and image ambiguity in the automated segmentation of medical images. ProSeg frameworks are designed to go beyond deterministic “single-mask” approaches, providing either personalized or distributional outputs that faithfully reflect the complex, multimodal annotation landscape encountered in clinical reality. Modern approaches draw on variational Bayes, hierarchical latent variable models, statistical-distance training objectives, and explicit disentangling of annotator preference from boundary uncertainty, delivering state-of-the-art accuracy and uncertainty quantification across heterogeneous datasets and tasks.

1. Modeling Principles and Probabilistic Factorizations

Contemporary ProSeg frameworks explicitly model both image-intrinsic ambiguity and expert-specific “preference” or bias. The core modeling distinction is between image uncertainty (variability in plausible contours due to imaging limitations) and inter-observer variability (systematic differences in annotator behavior). For example, the ProSeg model of (Liu et al., 30 Nov 2025) introduces two latent variable families: $z$ for image ambiguity and $\tau$ for expert preference, with the joint distribution: $$p(Y, R, X, \tau, Z) = p(Y|Z, \tau) \, p(X|Z) \, p(R|\tau) \, p(Z) \, p(\tau)$$ where $Y$ is the set of segmentations, $R$ the raters, $X$ the image, $(Z, \tau)$ the respective latents.

Alternative approaches such as hierarchical latent variable models (Baumgartner et al., 2019) or annotator-specific parameterizations (Schmidt et al., 2023, Liao et al., 2021) posit pixel- or rater-indexed latent codes that drive the conditional generation of plausible, expert-aligned segmentations. Deterministic baselines (e.g., soft-label averaging (Silva et al., 2021)) or models that collapse all labeler variability into aleatoric pixelwise confidence scores serve as important reference points, but do not capture the full stochastic structure of multi-rater data.

2. Variational Inference and Training Objectives

The dominant inferential machinery is variational autoencoding. The evidence lower bound (ELBO) objective is maximized with respect to both network parameters and (where applicable) annotator-specific or image-specific latent parameters. In (Liu et al., 30 Nov 2025), the ELBO for ProSeg takes the form: $$\mathcal{L} = L_{\mathrm{recon}} + L_{\mathrm{class}} + L_{\mathrm{seg}} + \mathrm{KL}(q(Z|X) \| p(Z)) + \mathrm{KL}(q(\tau|R) \| p(\tau))$$ where $L_{\mathrm{recon}}$ reconstructs the image, $L_{\mathrm{class}}$ is a cross-entropy for annotator identification, $L_{\mathrm{seg}}$ is segmentation loss, and the two KL terms regularize posteriors for ambiguity (Z) and preference (τ) against their respective priors.

Alternative objectives include statistical-distance-based losses (e.g., Hausdorff divergence, Sinkhorn OT, FID) for distributional calibration of samples to empirical annotator masks (Chatterjee et al., 2023). Soft-label deterministic baselines use pixelwise cross-entropy against averaged soft labels as in (Silva et al., 2021), with no explicit latent structure.

The sampling strategy during training—randomly selecting annotator masks per SGD iteration to expose posterior latents to the full support of expert variability—is a common motif enabling representation of multimodality (Ward et al., 6 Sep 2025, Baumgartner et al., 2019).

3. Latent Variable Parameterization and Network Architectures

Deep U-Net backbones (standard or EfficientNet/SAM hybrids) remain ubiquitous, with additional encoder/MLP modules for variational parameterization of latent spaces. The structure and role of latent variables distinguish model families:

• ProSeg (Liu et al., 30 Nov 2025): $z_i$ (image ambiguity, per expert) and $\tau_i$ (annotator preference, modeled as a Dirichlet) encoded by CNN/MLP stacks. Sampling $[\tau_i; z_i]$ modulates the segmentation predictor, enabling both personalization (conditional on a specific rater $r_i$) and diversity (sampling $\tau$ from the prior yields “virtual expert” segmentations).
• Probabilistic U-Net/PULASki (Chatterjee et al., 2023): single global $z$, concatenated to U-Net bottleneck, plus distributional statistical-distance losses to directly match sample statistics to multi-expert data.
• Probabilistic SAM (Ward et al., 6 Sep 2025): CVAE-style latent variable $z$ injected into prompt embedding pathway, all downstream segmentation masked through a frozen decoder, facilitating prompt-based multimodal mask generation.
• PADL (Liao et al., 2021): explicit parameterization of the annotator-specific deviation $\mu_r$ from consensus mask $\mu$, both with associated Gaussian “spread” $\sigma_r$, but without a global generative latent for yet-unseen annotators.

An illustrative table (editor’s condensation) of representative ProSeg formulations:

Method	Latent Variable(s)	Modeling Focus	Personalization	Diversity
ProSeg (Liu et al., 30 Nov 2025)	$z$ (ambig.), $\tau$ (preference)	Image uncertainty + expert bias	Yes	Yes
Prob. U-Net (Chatterjee et al., 2023)	$z$ (single global)	Sample-wise ambiguity	No	Yes
PADL (Liao et al., 2021)	$\mu,\ \mu_r,\ \sigma,\ \sigma_r$	Consensus + rater bias	Yes	Limited (no new raters)
PHiSeg (Baumgartner et al., 2019)	$\{z_i\}$ (multi-scale hierarchy)	Scale-wise ambiguity	No	Yes
Soft-label (Silva et al., 2021)	None	Averaged uncertainty	No	No

4. Evaluation Methodologies and Metrics

Quantitative evaluation universally adopts uncertainty- and personalization-aware metrics. The Generalized Energy Distance (GED)

$$\mathrm{GED} = 2\,\mathbb{E}_{Y,\hat Y}[d(Y,\hat Y)] - \mathbb{E}_{\hat Y,\hat Y'}[d(\hat Y,\hat Y')] - \mathbb{E}_{Y,Y'}[d(Y,Y')]$$

where $d(A,B)=1-\mathrm{IoU}(A,B)$, is a standard for measuring distributional alignment between sets of expert and model-generated masks (Liu et al., 30 Nov 2025, Ward et al., 6 Sep 2025, Chatterjee et al., 2023, Baumgartner et al., 2019). Soft Dice, Dice$_{\mathrm{max}}$, and Dice$_{\mathrm{match}}$ evaluate both average- and best-case similarity between model outputs and human annotations (Liu et al., 30 Nov 2025).

Personalization is directly measured by pairing model-sampled masks to specific ground-truth raters and assessing Dice or Cohen’s $\kappa$ (Schmidt et al., 2023).

Calibration properties are assessed by analyzing metric stability across confidence thresholds (Silva et al., 2021), or by using error–uncertainty correlation metrics (Baumgartner et al., 2019), while anatomical plausibility is frequently evaluated qualitatively (e.g., smoothness, boundary conformity in 3D (Chatterjee et al., 2023)).

5. Diversity, Personalization, and Limitations

A key innovation in (Liu et al., 30 Nov 2025) is explicit separation of diversity (via $z$ sampling: image-intrinsic ambiguity) from personalization (via $\tau$ sampling: rater-specific style). This dual-factor design enables, in a single unified ProSeg model, the flexible recovery of:

• Individual expert reconstructions (personalization, conditioning $\tau\sim q(\tau|r)$)
• Novel plausible masks from the diversity of clinical practice (diversification, sampling $\tau\sim p(\tau)$, $z\sim q(z|X)$)
• “Virtual experts” by sampling from the preference prior

Ablation studies confirm that removing either factor reduces both personalization (mean Dice) and diversity (GED). PADL (Liao et al., 2021) similarly models individual rater style but lacks a generative mechanism for unseen annotators.

Limitations persist. Soft-label “mean field” methods (Silva et al., 2021) cannot express diversity or bias, while models with a single global $z$ are liable to collapse diverse annotator distributions into a mean. Multi-modal or structured latent representations—mixture-of-Gaussians, hierarchical, or spatially-varying latent fields—are under ongoing investigation (Ward et al., 6 Sep 2025, Schmidt et al., 2023, Liu et al., 30 Nov 2025).

6. Representative Results and Comparative Performance

Empirical results on benchmarks such as LIDC-IDRI (CT lung nodules, 4 annotators), NPC (MRI nasopharyngeal carcinoma, 4 radiologists), and QUBIQ tasks consistently demonstrate the superiority of dual-latent ProSeg models over both deterministic and single-latent probabilistic baselines.

On LIDC-IDRI, ProSeg achieves lower GED ($\approx 0.115$), higher soft Dice ($91.53\%$), and higher mean Dice per rater ($90.25\%$) than Probabilistic U-Net, CM-Global/CM-Pixel, and other state-of-the-art baselines (Liu et al., 30 Nov 2025). On NPC, ProSeg achieves GED $0.227$ and mean Dice $82.07\%$, similarly surpassing all comparators. PULASki (Chatterjee et al., 2023) demonstrates computational efficiency and improved distributional calibration, particularly in 3D segmentation tasks with severe class imbalance.

On prompt-based segmentation using foundation models, Probabilistic SAM yields superior uncertainty-aware performance (e.g., GED $0.2910$, DSC $0.8255$) and plausible diversity without retraining the encoder/decoder backbone (Ward et al., 6 Sep 2025).

7. Extensions and Future Directions

Future work includes the development of models with spatially structured or hierarchical latent representations, improved multi-modal posteriors (e.g., mixture models or flows for better raterspace coverage), and integration of additional metadata (institution, scanner properties) into personalized priors (Liu et al., 30 Nov 2025, Schmidt et al., 2023). Efficient sampling at inference time, scalability to 3D imaging, and adaptability to prompt-based, interactive clinical workflows are active areas of exploration (Ward et al., 6 Sep 2025).

A plausible implication is that the separation of expert preference and image ambiguity within probabilistic segmentation pipelines is now a foundational design principle for robust clinical validation, personalized AI support tools, and uncertainty-aware risk modeling in medical imaging.