Papers
Topics
Authors
Recent
Search
2000 character limit reached

Modeling 3D Surface Manifolds with a Locally Conditioned Atlas

Published 11 Feb 2021 in cs.CV | (2102.05984v2)

Abstract: Recently proposed 3D object reconstruction methods represent a mesh with an atlas - a set of planar patches approximating the surface. However, their application in a real-world scenario is limited since the surfaces of reconstructed objects contain discontinuities, which degrades the quality of the final mesh. This is mainly caused by independent processing of individual patches, and in this work, we postulate to mitigate this limitation by preserving local consistency around patch vertices. To that end, we introduce a Locally Conditioned Atlas (LoCondA), a framework for representing a 3D object hierarchically in a generative model. Firstly, the model maps a point cloud of an object into a sphere. Secondly, by leveraging a spherical prior, we enforce the mapping to be locally consistent on the sphere and on the target object. This way, we can sample a mesh quad on that sphere and project it back onto the object's manifold. With LoCondA, we can produce topologically diverse objects while maintaining quads to be stitched together. We show that the proposed approach provides structurally coherent reconstructions while producing meshes of quality comparable to the competitors.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (49)
  1. Learning representations and generative models for 3d point clouds. arXiv preprint arXiv:1707.02392, 2017.
  2. Learning representations and generative models for 3d point clouds. In International conference on machine learning, pp.  40–49. PMLR, 2018.
  3. Sal: Sign agnostic learning of shapes from raw data. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.  2565–2574, 2020.
  4. Marr revisited: 2d-3d alignment via surface normal prediction. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp.  5965–5974, 2016.
  5. Learning to reconstruct texture-less deformable surfaces from a single view. In 2018 International Conference on 3D Vision (3DV), pp. 606–615. IEEE, 2018.
  6. Shape reconstruction by learning differentiable surface representations. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.  4716–4725, 2020.
  7. Wavegrad: Estimating gradients for waveform generation. arXiv preprint arXiv:2009.00713, 2020a.
  8. Learning implicit fields for generative shape modeling. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.  5939–5948, 2019.
  9. Bsp-net: Generating compact meshes via binary space partitioning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.  45–54, 2020b.
  10. 3d-r2n2: A unified approach for single and multi-view 3d object reconstruction. In European conference on computer vision, pp.  628–644. Springer, 2016.
  11. Cvxnet: Learnable convex decomposition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.  31–44, 2020a.
  12. Better patch stitching for parametric surface reconstruction. arXiv preprint arXiv:2010.07021, 2020b.
  13. Learning elementary structures for 3d shape generation and matching. arXiv preprint arXiv:1908.04725, 2019.
  14. Depth map prediction from a single image using a multi-scale deep network. arXiv preprint arXiv:1406.2283, 2014.
  15. A point set generation network for 3d object reconstruction from a single image. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp.  605–613, 2017.
  16. Ffjord: Free-form continuous dynamics for scalable reversible generative models. arXiv preprint arXiv:1810.01367, 2018.
  17. A papier-mâché approach to learning 3d surface generation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp.  216–224, 2018.
  18. Garnet: A two-stream network for fast and accurate 3d cloth draping. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp.  8739–8748, 2019.
  19. Hypernetworks. arXiv preprint arXiv:1609.09106, 2016.
  20. Hierarchical surface prediction for 3d object reconstruction. In 2017 International Conference on 3D Vision (3DV), pp. 412–420. IEEE, 2017.
  21. A survey of research on cloud robotics and automation. IEEE Transactions on automation science and engineering, 12(2):398–409, 2015.
  22. Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114, 2013.
  23. Cramer-wold auto-encoder. Journal of Machine Learning Research, 21, 2020.
  24. Point cloud gan. arXiv preprint arXiv:1810.05795, 2018.
  25. Occupancy networks: Learning 3d reconstruction in function space. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.  4460–4470, 2019.
  26. Shape unicode: A unified shape representation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.  3790–3799, 2019.
  27. Deepsdf: Learning continuous signed distance functions for shape representation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.  165–174, 2019.
  28. Learning unsupervised hierarchical part decomposition of 3d objects from a single rgb image. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.  1060–1070, 2020.
  29. Coupling explicit and implicit surface representations for generative 3d modeling. In European Conference on Computer Vision, pp.  667–683. Springer, 2020.
  30. Pointnet: Deep learning on point sets for 3d classification and segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp.  652–660, 2017a.
  31. Pointnet++: Deep hierarchical feature learning on point sets in a metric space. In Advances in neural information processing systems, pp. 5099–5108, 2017b.
  32. The earth mover’s distance as a metric for image retrieval. International journal of computer vision, 40(2):99–121, 2000.
  33. Stochastic maximum likelihood optimization via hypernetworks. arXiv preprint arXiv:1712.01141, 2017.
  34. Deep parametric shape predictions using distance fields. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.  561–570, 2020.
  35. Hypernetwork approach to generating point clouds. arXiv preprint arXiv:2003.00802, 2020a.
  36. Hyperflow: Representing 3d objects as surfaces. arXiv preprint arXiv:2006.08710, 2020b.
  37. Wasserstein auto-encoders. arXiv preprint arXiv:1711.01558, 2017.
  38. Tran, M.-P. 3d contour closing: A local operator based on chamfer distance transformation. 2013.
  39. Patch-based reconstruction of a textureless deformable 3d surface from a single rgb image. In Proceedings of the IEEE/CVF International Conference on Computer Vision Workshops, pp.  0–0, 2019.
  40. Pixel2mesh: Generating 3d mesh models from single rgb images. In Proceedings of the European Conference on Computer Vision (ECCV), pp.  52–67, 2018.
  41. Deep geometric prior for surface reconstruction. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.  10130–10139, 2019.
  42. Disn: Deep implicit surface network for high-quality single-view 3d reconstruction. arXiv preprint arXiv:1905.10711, 2019.
  43. Pixor: Real-time 3d object detection from point clouds. In Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, pp.  7652–7660, 2018a.
  44. Pointflow: 3d point cloud generation with continuous normalizing flows. In Proceedings of the IEEE International Conference on Computer Vision, pp.  4541–4550, 2019.
  45. Foldingnet: Point cloud auto-encoder via deep grid deformation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp.  206–215, 2018b.
  46. Front2back: Single view 3d shape reconstruction via front to back prediction. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.  531–540, 2020.
  47. Neural cages for detail-preserving 3d deformations. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.  75–83, 2020.
  48. Adversarial autoencoders for compact representations of 3d point clouds. Computer Vision and Image Understanding, 193:102921, 2020.
  49. Deep surface normal estimation with hierarchical rgb-d fusion. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.  6153–6162, 2019.
Citations (3)
View on Semantic Scholar

Summary

No one has generated a summary of this paper yet.

Sign Up to Summarize

Paper to Video (Beta)

No one has generated a video about this paper yet.

Sign Up to Generate All Videos Subscribe on YouTube

Whiteboard

No one has generated a whiteboard explanation for this paper yet.

Sign Up to Generate

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Generate Now

Continue Learning

We haven't generated follow-up questions for this paper yet.

Generate Now

Collections

Sign up for free to add this paper to one or more collections.

Sign Up

Tweets

Sign up for free to view the 1 tweet with 0 likes about this paper.

Sign Up for Free