Papers
Topics
Authors
Recent
Search
2000 character limit reached

GGAvatar: Geometric Adjustment of Gaussian Head Avatar

Published 20 May 2024 in cs.CV | (2405.11993v1)

Abstract: We propose GGAvatar, a novel 3D avatar representation designed to robustly model dynamic head avatars with complex identities and deformations. GGAvatar employs a coarse-to-fine structure, featuring two core modules: Neutral Gaussian Initialization Module and Geometry Morph Adjuster. Neutral Gaussian Initialization Module pairs Gaussian primitives with deformable triangular meshes, employing an adaptive density control strategy to model the geometric structure of the target subject with neutral expressions. Geometry Morph Adjuster introduces deformation bases for each Gaussian in global space, creating fine-grained low-dimensional representations of deformation behaviors to address the Linear Blend Skinning formula's limitations effectively. Extensive experiments show that GGAvatar can produce high-fidelity renderings, outperforming state-of-the-art methods in visual quality and quantitative metrics.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (41)
  1. Flame-in-nerf: Neural control of radiance fields for free view face animation. In 2023 IEEE 17th International Conference on Automatic Face and Gesture Recognition (FG) (2023), IEEE, pp. 1–8.
  2. Rignerf: Fully controllable neural 3d portraits. In Proceedings of the IEEE/CVF conference on Computer Vision and Pattern Recognition (2022), pp. 20364–20373.
  3. Generative neural articulated radiance fields. Advances in Neural Information Processing Systems 35 (2022), 19900–19916.
  4. Blanz V., Vetter T.: A morphable model for the synthesis of 3d faces. In Proceedings of the 26th annual conference on Computer graphics and interactive techniques - SIGGRAPH ’99 (Jan 1999). URL: http://dx.doi.org/10.1145/311535.311556, doi:10.1145/311535.311556.
  5. Monogaussianavatar: Monocular gaussian point-based head avatar. arXiv preprint arXiv:2312.04558 (2023).
  6. Facewarehouse: A 3d facial expression database for visual computing. IEEE Transactions on Visualization and Computer Graphics 20, 3 (2013), 413–425.
  7. Bakedavatar: Baking neural fields for real-time head avatar synthesis. ACM Transactions on Graphics (TOG) 42, 6 (2023), 1–17.
  8. Learning an animatable detailed 3d face model from in-the-wild images. ACM Transactions on Graphics (ToG) 40, 4 (2021), 1–13.
  9. K-planes: Explicit radiance fields in space, time, and appearance. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (2023), pp. 12479–12488.
  10. Morphable face models-an open framework. In 2018 13th IEEE international conference on automatic face & gesture recognition (FG 2018) (2018), IEEE, pp. 75–82.
  11. Neural head avatars from monocular rgb videos. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (2022), pp. 18653–18664.
  12. Dynamic view synthesis from dynamic monocular video. In Proceedings of the IEEE/CVF International Conference on Computer Vision (2021), pp. 5712–5721.
  13. Dynamic neural radiance fields for monocular 4d facial avatar reconstruction. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (2021), pp. 8649–8658.
  14. Reconstructing personalized semantic facial nerf models from monocular video. ACM Transactions on Graphics (TOG) 41, 6 (2022), 1–12.
  15. Headnerf: A real-time nerf-based parametric head model. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (2022), pp. 20374–20384.
  16. Perceptual losses for real-time style transfer and super-resolution. In Computer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11-14, 2016, Proceedings, Part II 14 (2016), Springer, pp. 694–711.
  17. Hugs: Human gaussian splats. arXiv preprint arXiv:2311.17910 (2023).
  18. Deep video portraits. ACM transactions on graphics (TOG) 37, 4 (2018), 1–14.
  19. 3d gaussian splatting for real-time radiance field rendering. ACM Transactions on Graphics 42, 4 (2023), 1–14.
  20. Realistic one-shot mesh-based head avatars. In European Conference on Computer Vision (2022), Springer, pp. 345–362.
  21. Learning a model of facial shape and expression from 4d scans. ACM Trans. Graph. 36, 6 (2017), 194–1.
  22. Semantic-aware implicit neural audio-driven video portrait generation. In European conference on computer vision (2022), Springer, pp. 106–125.
  23. Robust high-resolution video matting with temporal guidance. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (2022), pp. 238–247.
  24. Nerf: Representing scenes as neural radiance fields for view synthesis. Communications of the ACM 65, 1 (2021), 99–106.
  25. Deepsdf: Learning continuous signed distance functions for shape representation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (2019), pp. 165–174.
  26. A 3d face model for pose and illumination invariant face recognition. In 2009 sixth IEEE international conference on advanced video and signal based surveillance (2009), Ieee, pp. 296–301.
  27. Face reconstruction from skull shapes and physical attributes. In Pattern Recognition: 31st DAGM Symposium, Jena, Germany, September 9-11, 2009. Proceedings 31 (2009), Springer, pp. 232–241.
  28. Gaussianavatars: Photorealistic head avatars with rigged 3d gaussians. arXiv preprint arXiv:2312.02069 (2023).
  29. Splattingavatar: Realistic real-time human avatars with mesh-embedded gaussian splatting. arXiv preprint arXiv:2403.05087 (2024).
  30. Neural voice puppetry: Audio-driven facial reenactment. In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XVI 16 (2020), Springer, pp. 716–731.
  31. Deferred neural rendering: Image synthesis using neural textures. Acm Transactions on Graphics (TOG) 38, 4 (2019), 1–12.
  32. Learning compositional radiance fields of dynamic human heads. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (2021), pp. 5704–5713.
  33. Prior-guided multi-view 3d head reconstruction. IEEE Transactions on Multimedia 24 (2021), 4028–4040.
  34. Flashavatar: High-fidelity digital avatar rendering at 300fps. arXiv preprint arXiv:2312.02214 (2023).
  35. Animatable 3d gaussians for high-fidelity synthesis of human motions. arXiv preprint arXiv:2311.13404 (2023).
  36. Im avatar: Implicit morphable head avatars from videos. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (2022), pp. 13545–13555.
  37. Psavatar: A point-based morphable shape model for real-time head avatar creation with 3d gaussian splatting. arXiv preprint arXiv:2401.12900 (2024).
  38. Towards metrical reconstruction of human faces. In European Conference on Computer Vision (2022), Springer, pp. 250–269.
  39. The unreasonable effectiveness of deep features as a perceptual metric. In 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition (Jun 2018). URL: http://dx.doi.org/10.1109/cvpr.2018.00068, doi:10.1109/cvpr.2018.00068.
  40. Havatar: High-fidelity head avatar via facial model conditioned neural radiance field. ACM Transactions on Graphics 43, 1 (2023), 1–16.
  41. Pointavatar: Deformable point-based head avatars from videos. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (2023), pp. 21057–21067.
Citations (1)
View on Semantic Scholar

Summary

No one has generated a summary of this paper yet.

Sign Up to Summarize

Paper to Video (Beta)

No one has generated a video about this paper yet.

Sign Up to Generate All Videos Subscribe on YouTube

Whiteboard

No one has generated a whiteboard explanation for this paper yet.

Sign Up to Generate

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Generate Now

Continue Learning

We haven't generated follow-up questions for this paper yet.

Generate Now

Collections

Sign up for free to add this paper to one or more collections.

Sign Up