Fuzzy-Conditioned Diffusion and Diffusion Projection Attention Applied to Facial Image Correction
Abstract: Image diffusion has recently shown remarkable performance in image synthesis and implicitly as an image prior. Such a prior has been used with conditioning to solve the inpainting problem, but only supporting binary user-based conditioning. We derive a fuzzy-conditioned diffusion, where implicit diffusion priors can be exploited with controllable strength. Our fuzzy conditioning can be applied pixel-wise, enabling the modification of different image components to varying degrees. Additionally, we propose an application to facial image correction, where we combine our fuzzy-conditioned diffusion with diffusion-derived attention maps. Our map estimates the degree of anomaly, and we obtain it by projecting on the diffusion space. We show how our approach also leads to interpretable and autonomous facial image correction.
- “Deep unsupervised learning using nonequilibrium thermodynamics,” in International Conference on Machine Learning (ICML), 2015.
- “Denoising diffusion probabilistic models,” in Advances in Neural Information Processing Systems (NeurIPS), 2020.
- “VAEs meet diffusion models: Efficient and high-fidelity generation,” in Advances in Neural Information Processing Systems (NeurIPS) Workshop, 2021.
- “DiffuseVAE: Efficient, controllable and high-fidelity generation from low-dimensional latents,” arXiv preprint arXiv:2201.00308, 2022.
- “Score-based generative neural networks for large-scale optimal transport,” in Advances in Neural Information Processing Systems (NeurIPS), 2021.
- Y. Benny and L. Wolf, “Dynamic dual-output diffusion models,” in IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022.
- “Generating high fidelity data from low-density regions using diffusion models,” in IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022.
- “Diffusion models in vision: A survey,” IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2022.
- “Diffusion posterior sampling for general noisy inverse problems,” arXiv preprint arXiv:2209.14687, 2022.
- M. El Helou and S. Süsstrunk, “BIGPrior: Toward decoupling learned prior hallucination and data fidelity in image restoration,” IEEE Transactions on Image Processing (TIP), 2022.
- “UNIT-DDPM: Unpaired image translation with denoising diffusion probabilistic models,” arXiv preprint arXiv:2104.05358, 2021.
- “Palette: Image-to-image diffusion models,” in ACM SIGGRAPH, 2022.
- “Image super-resolution via iterative refinement,” IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2022.
- “Photorealistic text-to-image diffusion models with deep language understanding,” arXiv preprint arXiv:2205.11487, 2022.
- “A structure-guided diffusion model for large-hole diverse image completion,” arXiv preprint arXiv:2211.10437, 2022.
- “SDEdit: Guided image synthesis and editing with stochastic differential equations,” in International Conference on Learning Representations (ICLR), 2021.
- “RePaint: Inpainting using denoising diffusion probabilistic models,” in IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022.
- M. El Helou and S. Süsstrunk, “Blind universal Bayesian image denoising with Gaussian noise level learning,” IEEE Transactions on Image Processing (TIP), 2020.
- M. El Helou, Deep Image Restoration: Between Data Fidelity and Learned Priors, Ph.D. thesis, EPFL, 2021.
- “Diffusion denoising process for perceptron bias in out-of-distribution detection,” arXiv preprint arXiv:2211.11255, 2022.
- “AnoDDPM: Anomaly detection with denoising diffusion probabilistic models using simplex noise,” in IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 2022.
- “Denoising diffusion models for out-of-distribution detection,” arXiv preprint arXiv:2211.07740, 2022.
- “Memorizing normality to detect anomaly: Memory-augmented deep autoencoder for unsupervised anomaly detection,” in International Conference on Computer Vision (ICCV), 2019.
- “Deep learning face attributes in the wild,” in IEEE International Conference on Computer Vision (ICCV), 2015.
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
Top Community Prompts
Collections
Sign up for free to add this paper to one or more collections.