Pseudo Ground-Truth Diffusion Techniques

• Pseudo ground-truth diffusion is a method that uses surrogate teacher signals or self-supervised annotations to replace unavailable ground truth in training diffusion models.
• It integrates noise injection, masking, and conditional denoising to align pseudo labels with model predictions through distillation and reconstruction losses.
• Empirical results in tasks like monocular depth estimation and face-swapping show improved performance and resilience to annotation noise over traditional methods.

Pseudo ground-truth diffusion refers to methodologies in diffusion models that leverage surrogate signals, typically generated by teacher networks or self-supervised clustering algorithms, in place of inaccessible or expensive ground-truth data. This paradigm allows diffusion models to be trained on challenging tasks such as monocular depth estimation, face swapping, and arbitrary image generation, where annotated ground truth is unavailable or costly to obtain. The pseudo ground-truth serves both as the target for noise injection in the forward diffusion process and as the reference for denoising and distillation objectives during learning and inference.

1. Mathematical Foundations of Pseudo Ground-Truth Diffusion

The pseudo ground-truth diffusion process is built upon the standard formulation of denoising diffusion probabilistic models (DDPM/DDIM). Given a “clean” signal $x_0$, diffusion steps $t = 1 \dots T$ progressively corrupt $x_0$ with Gaussian noise, controlled by a variance schedule $\beta_t$:

• $\alpha_t = 1 - \beta_t$
• $\bar{\alpha}_t = \prod_{n=1}^t \alpha_n$

At each step $t$, the forward process is:

$$q(x_t | x_0) = \mathcal{N}(x_t; \sqrt{\bar{\alpha}_t} \, x_0, (1 - \bar{\alpha}_t)I)$$

Sampling noise $\epsilon \sim \mathcal{N}(0, I)$, the noised version is $$x_t = \sqrt{\bar{\alpha}_t} x_0 + \sqrt{1 - \bar{\alpha}_t} \epsilon$$.

In pseudo ground-truth diffusion, the “clean” target $t = 1 \dots T$0 is replaced by a pseudo ground-truth $t = 1 \dots T$1 generated by a teacher model or by self-annotation procedures, as in MonoDiffusion (Shao et al., 2023) and Self-Guided Diffusion (Hu et al., 2022). The denoising model, typically a U-Net, is conditioned on additional context signals and trained to predict the injected noise $t = 1 \dots T$2 or the clean signal via learned parameterizations.

2. Generation and Filtering of Pseudo Ground-Truth

When true ground-truth data is absent, pseudo ground-truth must be constructed through alternative strategies:

• Teacher Model Prediction: MonoDiffusion trains a self-supervised teacher (e.g., Lite-Mono) to predict depth from monocular input. Pseudo ground-truth $t = 1 \dots T$3 for each pixel $t = 1 \dots T$4 is sourced from teacher outputs.
• Self-Supervised Annotation: In Self-Guided Diffusion Models, self-annotations $t = 1 \dots T$5 (pseudo-labels, masks, bounding boxes) are derived from feature clustering or specific unsupervised algorithms like k-means, LOST, or STEGO applied to deep features $t = 1 \dots T$6.
• Filtering: Quality control is imposed via masking mechanisms such as multi-view consistency checks $t = 1 \dots T$7 (Shao et al., 2023), which exclude unreliable teacher predictions from loss computation and supervision.

This approach generalizes across modalities: depth estimation, semantic image labeling, and local attribute transfer can all utilize pseudo ground-truth, provided the signals are sufficiently representative and robust against annotation noise.

3. Integrating Pseudo Ground-Truth into Diffusion Training

Diffusion models exploit pseudo ground-truth during the entire training workflow. The integration follows these typical steps:

Step	Description	Example Reference
Pseudo GT generation	Obtain $t = 1 \dots T$8 from teacher network or self-annotation	(Shao et al., 2023, Hu et al., 2022)
Noise injection	Apply DDPM forward process to $t = 1 \dots T$9	(Shao et al., 2023)
Masking/filtering	Compute masks $x_0$0 for valid supervision points	(Shao et al., 2023)
Conditional denoising	Train denoiser $x_0$1 on noisy pseudo ground-truth	(Shao et al., 2023, Hu et al., 2022)
Distillation/losses	Apply knowledge distillation, denoising, reconstruction losses	(Shao et al., 2023, Kang et al., 21 Jan 2026)

For each minibatch, the training loop samples a diffusion step $x_0$2, injects Gaussian noise, computes the context/masked condition, and updates model parameters via backpropagation over composite losses. Typical objective terms include:

• Photometric losses on synthesized outputs (when applicable)
• Distillation losses aligning predictions with the pseudo ground-truth
• Denoising objectives matching noise predictions to the injected noise
• Masked-reconstruction terms to inpaint missing regions or filtered pixels

Hyperparameters such as mask ratio, learning rate, and loss weights are empirically tuned for stability and convergence.

4. Architectural Conditioning and Masked Context Mechanisms

Conditioning the diffusion model on relevant context is critical to the effectiveness of pseudo ground-truth diffusion.

Masked Visual Conditioning (Shao et al., 2023)

• Multi-scale encoder feature maps $x_0$3 are masked by random binary masks $x_0$4 with a typical fill ratio $x_0$5.
• Masked features $x_0$6 are aggregated to full-resolution tensors $x_0$7 via convolution and upsampling.
• The U-Net denoiser receives $x_0$8 (noisy depth), time/noise embeddings, and $x_0$9. Skip connections and concatenations propagate masked context, training the model to robustly “inpaint” and infer structure under partial observation.

Conditional Guidance via Pseudo Annotation (Hu et al., 2022)

• Self-generated annotations $\beta_t$0 (clusters, boxes, masks) are concatenated to diffusion block embeddings, providing semantic or local conditioning signals.
• Classifier-free guidance is implemented by mixing predictions with and without conditioning, offering flexible generative control.

A plausible implication is that context-aware conditioning can regularize denoising in regions where the pseudo ground-truth is most reliable, and inpainting elsewhere, thereby reducing the impact of teacher or annotation noise on the final model outputs.

5. Loss Functions and Distillation Strategies

Pseudo ground-truth diffusion models leverage several loss types:

• Knowledge Distillation Loss:

$\beta_t$1

Used in MonoDiffusion for aligning student predictions to filtered teacher outputs, with mask $\beta_t$2 (Shao et al., 2023).

• Denoising Loss:

Objectives such as $\beta_t$3 match predicted noise to sampled noise $\beta_t$4 (Shao et al., 2023, Hu et al., 2022).

• Pseudo-Label Supervision:

In face-swapping, APPLE trains the student on triplets $\beta_t$5 using both pixel-level pseudo-label losses and identity/attribute separation losses (Kang et al., 21 Jan 2026).

• Reconstruction and Photometric Losses:

Additional loss terms penalize discrepancies in masked or reconstructed regions, or measure appearance consistency using perceptual metrics.

Loss weighting (e.g., $\beta_t$6 for photometric, $\beta_t$7 for distill, $\beta_t$8 for denoising (Shao et al., 2023)) is tuned to balance supervision signals.

6. Empirical Results and Impact

Pseudo ground-truth diffusion frameworks have demonstrated substantial empirical improvements over prior baselines reliant on direct supervision or standard self-supervision:

• MonoDiffusion achieves Abs Rel $\beta_t$9 on KITTI depth benchmarks, surpassing Lite-Mono baseline ($\alpha_t = 1 - \beta_t$0), with further gains ($\alpha_t = 1 - \beta_t$1) using larger backbones (Shao et al., 2023).
• Ablation studies indicate that pseudo–ground-truth diffusion paired with distillation and masked visual condition yields the best performance; naive self-diffusion without pseudo ground-truth fails to converge.
• In Self-Guided Diffusion, FID metrics for self-labeled guidance ($\alpha_t = 1 - \beta_t$2) exceed those for ground-truth label guidance and unguided models ($\alpha_t = 1 - \beta_t$3) (Hu et al., 2022).
• APPLE face-swapping achieves superior attribute preservation and identity transfer (FID $\alpha_t = 1 - \beta_t$4, pose error $\alpha_t = 1 - \beta_t$5) compared to previous methods (Kang et al., 21 Jan 2026).
• Robustness to noise and imperfect pseudo ground-truth has been observed, provided filtering and masking mechanisms are in place.

This suggests that the pseudo ground-truth diffusion approach, by introducing a continuum of noisy targets and coupling denoising with distillation and context reasoning, offers a principled alternative for self-supervised training where ground-truth data is inaccessible.

7. Connections, Limitations, and Prospects

Pseudo ground-truth diffusion techniques are closely related to broader self-supervised learning and teacher-student regimes. They circumvent annotation bottlenecks by producing surrogate supervision underpinned by physical consistency checks, clustering in feature space, or data-driven hallucinations. Key limitations documented include:

• Upper bounds related to the quality of pseudo ground-truth and annotation algorithms (feature extractor, clustering fidelity, teacher supervision) (Hu et al., 2022).
• Computational overhead for generating and filtering pseudo annotations.
• Potential failure modes if teacher predictions are highly biased or cluster assignments do not reflect true semantic structure.

Future directions include scaling up pseudo ground-truth diffusion to web-scale datasets, integrating multi-modal self-supervision, and jointly evolving the pseudo annotation mechanisms with the diffusion model itself. Such advances are expected to further enhance model generalizability and robustness to annotation noise, continuing the trajectory of research in self-supervised generative modeling.