Papers
Topics
Authors
Recent
Search
2000 character limit reached

Inferring Neural Signed Distance Functions by Overfitting on Single Noisy Point Clouds through Finetuning Data-Driven based Priors

Published 25 Oct 2024 in cs.CV | (2410.19680v1)

Abstract: It is important to estimate an accurate signed distance function (SDF) from a point cloud in many computer vision applications. The latest methods learn neural SDFs using either a data-driven based or an overfitting-based strategy. However, these two kinds of methods are with either poor generalization or slow convergence, which limits their capability under challenging scenarios like highly noisy point clouds. To resolve this issue, we propose a method to promote pros of both data-driven based and overfitting-based methods for better generalization, faster inference, and higher accuracy in learning neural SDFs. We introduce a novel statistical reasoning algorithm in local regions which is able to finetune data-driven based priors without signed distance supervision, clean point cloud, or point normals. This helps our method start with a good initialization, and converge to a minimum in a much faster way. Our numerical and visual comparisons with the state-of-the-art methods show our superiority over these methods in surface reconstruction and point cloud denoising on widely used shape and scene benchmarks. The code is available at https://github.com/chenchao15/LocalN2NM.

Authors (3)
Definition Search Book Streamline Icon: https://streamlinehq.com
References (106)
  1. SAL: Sign agnostic learning of shapes from raw data. In IEEE Conference on Computer Vision and Pattern Recognition, 2020.
  2. SALD: sign agnostic learning with derivatives. In International Conference on Learning Representations, 2021.
  3. DiGS: Divergence guided shape implicit neural representation for unoriented point clouds. In IEEE Conference on Computer Vision and Pattern Recognition, 2022.
  4. DiGS : Divergence guided shape implicit neural representation for unoriented point clouds. CoRR, abs/2106.10811, 2021.
  5. Dynamic FAUST: Registering human bodies in motion. In IEEE Computer Vision and Pattern Recognition, 2017.
  6. POCO: Point convolution for surface reconstruction. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 6302–6314, June 2022.
  7. Poco: Point convolution for surface reconstruction. In IEEE Conference on Computer Vision and Pattern Recognition, 2022.
  8. nuscenes: A multimodal dataset for autonomous driving. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020.
  9. Learning gradient fields for shape generation. In European Conference on Computer Vision, 2020.
  10. Total denoising: Unsupervised learning of 3d point cloud cleaning. In IEEE International Conference on Computer Vision, pages 52–60, 2019.
  11. Deep local shapes: Learning local SDF priors for detailed 3D reconstruction. In European Conference on Computer Vision, volume 12374, pages 608–625, 2020.
  12. ShapeNet: An Information-Rich 3D Model Repository. Technical Report arXiv:1512.03012 [cs.GR], Stanford University — Princeton University — Toyota Technological Institute at Chicago, 2015.
  13. Unsupervised inference of signed distance functions from single sparse point clouds without learning priors. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 17712–17723, 2023.
  14. Unsupervised learning of fine structure generation for 3D point clouds by 2D projections matching. In IEEE International Conference on Computer Vision, 2021.
  15. Latent partition implicit with surface codes for 3d representation. In European Conference on Computer Vision, 2022.
  16. GridPull: Towards scalability in learning implicit representations from 3d point clouds. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 18322–18334, 2023.
  17. Learning implicit fields for generative shape modeling. IEEE Conference on Computer Vision and Pattern Recognition, 2019.
  18. Neural unsigned distance fields for implicit function learning. arXiv, 2010.13938, 2020.
  19. GenSDF: Two-stage learning of generalizable signed distance functions. In Advances in Neural Information Processing Systems, pages 24905–24919, 2022.
  20. 3D-r2n2: A unified approach for single and multi-view 3d object reconstruction. In Bastian Leibe, Jiri Matas, Nicu Sebe, and Max Welling, editors, European Conference on Computer Vision, volume 9912, pages 628–644, 2016.
  21. Neural Poisson: Indicator functions for neural fields. arXiv preprint arXiv:2211.14249, 2022.
  22. Points2Surf: Learning implicit surfaces from point clouds. In European Conference on Computer Vision, 2020.
  23. DUDF: Differentiable unsigned distance fields with hyperbolic scaling. arXiv preprint arXiv:2402.08876, 2024.
  24. A point set generation network for 3D object reconstruction from a single image. In 2017 IEEE Conference on Computer Vision and Pattern Recognition, pages 2463–2471, 2017.
  25. Geo-Neus: Geometry-consistent neural implicit surfaces learning for multi-view reconstruction. 2022.
  26. Are we ready for autonomous driving? the kitti vision benchmark suite. In Computer Vision and Pattern Recognition, 2012.
  27. Learning shape templates with structured implicit functions. In International Conference on Computer Vision, 2019.
  28. Implicit geometric regularization for learning shapes. In International Conference on Machine Learning, volume 119 of Proceedings of Machine Learning Research, pages 3789–3799, 2020.
  29. DRWR: A differentiable renderer without rendering for unsupervised 3D structure learning from silhouette images. In International Conference on Machine Learning, 2020.
  30. ShapeCaptioner: Generative caption network for 3D shapes by learning a mapping from parts detected in multiple views to sentences. In ACM International Conference on Multimedia, 2020.
  31. Hierarchical view predictor: Unsupervised 3d global feature learning through hierarchical prediction among unordered views. In Proceedings of the 29th ACM International Conference on Multimedia, pages 3862––3871, 2021.
  32. Point2mesh: a self-prior for deformable meshes. ACM Transactions on Graphics, 39(4):126, 2020.
  33. Learning neural implicit through volume rendering with attentive depth fusion priors. In Advances in Neural Information Processing Systems, 2023.
  34. A neural galerkin solver for accurate surface reconstruction. ACM Trans. Graph., 41(6), 2022.
  35. Neural kernel surface reconstruction. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4369–4379, 2023.
  36. Learning occupancy function from point clouds for surface reconstruction. arXiv, 2010.11378, 2020.
  37. Local implicit grid representations for 3D scenes. In IEEE Conference on Computer Vision and Pattern Recognition, 2020.
  38. SDFDiff: Differentiable rendering of signed distance fields for 3D shape optimization. In IEEE Conference on Computer Vision and Pattern Recognition, 2020.
  39. Screened poisson surface reconstruction. ACM Transactions on Graphics, 32(3):29:1–29:13, 2013.
  40. Noise2noise: Learning image restoration without clean data. In Jennifer G. Dy and Andreas Krause, editors, International Conference on Machine Learning, volume 80, pages 2971–2980, 2018.
  41. SDF-SRN: Learning signed distance 3D object reconstruction from static images. In Advances in Neural Information Processing Systems, 2020.
  42. Point2Skeleton: Learning skeletal representations from point clouds. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4277–4286, 2021.
  43. Meshing point clouds with predicted intrinsic-extrinsic ratio guidance. In European Conference on Computer vision, 2020.
  44. DIST: Rendering deep implicit signed distance function with differentiable sphere tracing. In IEEE Conference on Computer Vision and Pattern Recognition, 2020.
  45. Deep implicit moving least-squares functions for 3D reconstruction. In IEEE Conference on Computer Vision and Pattern Recognition, 2021.
  46. Learning to infer implicit surfaces without 3D supervision. In Advances in Neural Information Processing Systems, 2019.
  47. Marching cubes: A high resolution 3D surface construction algorithm. Computer Graphics, 21(4):163–169, 1987.
  48. Differentiable manifold reconstruction for point cloud denoising. In ACM International Conference on Multimedia, pages 1330–1338. ACM, 2020.
  49. Score-based point cloud denoising. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 4583–4592, 2021.
  50. Neural-pull: Learning signed distance functions from point clouds by learning to pull space onto surfaces. In International Conference on Machine Learning, 2021.
  51. Reconstructing surfaces for sparse point clouds with on-surface priors. In IEEE Conference on Computer Vision and Pattern Recognition, pages 6305–6315, 2022.
  52. Learning signed distance functions from noisy 3D point clouds via noise to noise mapping. In International Conference on Machine Learning, pages 23338–23357. PMLR, 2023.
  53. Reconstructing surfaces for sparse point clouds with on-surface priors. In IEEE Conference on Computer Vision and Pattern Recognition, 2022.
  54. Surface reconstruction from point clouds by learning predictive context priors. In IEEE Conference on Computer Vision and Pattern Recognition, pages 6316–6327, 2022.
  55. Surface reconstruction from point clouds by learning predictive context priors. In IEEE Conference on Computer Vision and Pattern Recognition, 2022.
  56. Towards better gradient consistency for neural signed distance functions via level set alignment. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 17724–17734, 2023.
  57. Denoising point clouds in latent space via graph convolution and invertible neural network. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 5768–5777, June 2024.
  58. ACORN: adaptive coordinate networks for neural scene representation. CoRR, abs/2105.02788, 2021.
  59. Occupancy networks: Learning 3D reconstruction in function space. In IEEE Conference on Computer Vision and Pattern Recognition, 2019.
  60. SSRNet: Scalable 3D surface reconstruction network. In IEEE Conference on Computer Vision and Pattern Recognition, 2020.
  61. Deep level sets: Implicit surface representations for 3D shape inference. CoRR, abs/1901.06802, 2019.
  62. NeRF: Representing scenes as neural radiance fields for view synthesis. In European Conference on Computer Vision, 2020.
  63. Differentiable volumetric rendering: Learning implicit 3D representations without 3D supervision. In IEEE Conference on Computer Vision and Pattern Recognition, 2020.
  64. UNISURF: Unifying neural implicit surfaces and radiance fields for multi-view reconstruction. In International Conference on Computer Vision, 2021.
  65. Few ’zero level set’-shot learning of shape signed distance functions in feature space. In European Conference on Computer Vision, 2022.
  66. Robustifying generalizable implicit shape networks with a tunable non-parametric model. In Advances in Neural Information Processing Systems, 2023.
  67. Mixing-denoising generalizable occupancy networks. In International Conference on 3D Vision, 2024.
  68. Unsupervised occupancy learning from sparse point cloud. In IEEE Conference on Computer Vision and Pattern Recognition, 2024.
  69. DeepSDF: Learning continuous signed distance functions for shape representation. In IEEE Conference on Computer Vision and Pattern Recognition, 2019.
  70. Shape as points: A differentiable poisson solver. In Advances in Neural Information Processing Systems, 2021.
  71. Convolutional occupancy networks. In European Conference on Computer Vision, volume 12348, pages 523–540, 2020.
  72. Learning graph-convolutional representations for point cloud denoising. In European Conference on Computer Vision, volume 12365, pages 103–118, 2020.
  73. Pointcleannet: Learning to denoise and remove outliers from dense point clouds. Computer Graphics Forum, 39(1):185–203, 2020.
  74. Sharf: Shape-conditioned radiance fields from a single view. In International Conference on Machine Learning, 2021.
  75. Paris-rue-madame database - A 3D mobile laser scanner dataset for benchmarking urban detection, segmentation and classification methods. In International Conference on Pattern Recognition Applications and Methods, pages 819–824, 2014.
  76. DITTO: Dual and integrated latent topologies for implicit 3d reconstruction. arXiv preprint arXiv:2403.05005, 2024.
  77. Implicit neural representations with periodic activation functions. In Advances in Neural Information Processing Systems, 2020.
  78. Scene representation networks: Continuous 3D-structure-aware neural scene representations. In Advances in Neural Information Processing Systems, 2019.
  79. Convolutional occupancy networks. In European Conference on Computer Vision, 2020.
  80. Neural geometric level of detail: Real-time rendering with implicit 3D shapes. In IEEE Conference on Computer Vision and Pattern Recognition, 2021.
  81. SA-ConvONet: Sign-agnostic optimization of convolutional occupancy networks. In Proceedings of the IEEE/CVF International Conference on Computer Vision, 2021.
  82. What do single-view 3D reconstruction networks learn? In The IEEE Conference on Computer Vision and Pattern Recognition, 2019.
  83. PatchNets: Patch-Based Generalizable Deep Implicit 3D Shape Representations. European Conference on Computer Vision, 2020.
  84. Differentiable signed distance function rendering. ACM Transactions on Graphics, 41(4):125:1–125:18, 2022.
  85. NeuRIS: Neural reconstruction of indoor scenes using normal priors. In European Conference on Computer Vision, 2022.
  86. NeuS: Learning neural implicit surfaces by volume rendering for multi-view reconstruction. In Advances in Neural Information Processing Systems, pages 27171–27183, 2021.
  87. Deep octree-based cnns with output-guided skip connections for 3d shape and scene completion. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pages 266–267, 2020.
  88. Aligning gradient and hessian for neural signed distance function. In Advances in Neural Information Processing Systems, volume 36, pages 63515–63528, 2023.
  89. HF-NeuS: Improved surface reconstruction using high-frequency details. 2022.
  90. Alto: Alternating latent topologies for implicit 3d reconstruction. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 259–270, 2023.
  91. Neural fields as learnable kernels for 3D reconstruction. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 18500–18510, 2022.
  92. Deep geometric prior for surface reconstruction. In IEEE Conference on Computer Vision and Pattern Recognition, 2019.
  93. Neural splines: Fitting 3D surfaces with infinitely-wide neural networks. In IEEE Conference on Computer Vision and Pattern Recognition, pages 9949–9958, 2021.
  94. DFR: differentiable function rendering for learning 3D generation from images. Computer Graphics Forum, 39(5):241–252, 2020.
  95. SnowflakeNet: Point cloud completion by snowflake point deconvolution with skip-transformer. In IEEE International Conference on Computer Vision, 2021.
  96. StEik: Stabilizing the optimization of neural signed distance functions and finer shape representation. In Advances in Neural Information Processing Systems, 2023.
  97. Volume rendering of neural implicit surfaces. In Advances in Neural Information Processing Systems, 2021.
  98. Multiview neural surface reconstruction by disentangling geometry and appearance. Advances in Neural Information Processing Systems, 33, 2020.
  99. Iso-Points: Optimizing neural implicit surfaces with hybrid representations. CoRR, abs/2012.06434, 2020.
  100. MonoSDF: Exploring monocular geometric cues for neural implicit surface reconstruction. 2022.
  101. Autolabeling 3D objects with differentiable rendering of sdf shape priors. In IEEE Conference on Computer Vision and Pattern Recognition, 2020.
  102. Sign-agnostic implicit learning of surface self-similarities for shape modeling and reconstruction from raw point clouds. CoRR, abs/2012.07498, 2020.
  103. Learning a more continuous zero level set in unsigned distance fields through level set projection. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 3181–3192, 2023.
  104. Learning consistency-aware unsigned distance functions progressively from raw point clouds. In Advances in Neural Information Processing Systems, 2022.
  105. Dense scene reconstruction with points of interest. ACM Transactions on Graphics, 32(4):112:1–112:8, 2013.
  106. Ssp: Semi-signed prioritized neural fitting for surface reconstruction from unoriented point clouds. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), pages 3769–3778, 2024.
Citations (2)
View on Semantic Scholar

Summary

No one has generated a summary of this paper yet.

Sign Up to Summarize

Paper to Video (Beta)

No one has generated a video about this paper yet.

Sign Up to Generate All Videos Subscribe on YouTube

Whiteboard

No one has generated a whiteboard explanation for this paper yet.

Sign Up to Generate

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Generate Now

Continue Learning

We haven't generated follow-up questions for this paper yet.

Generate Now

Collections

Sign up for free to add this paper to one or more collections.

Sign Up