2000 character limit reached
Spiking NeRF: Representing the Real-World Geometry by a Discontinuous Representation
Published 15 Nov 2023 in cs.CV | (2311.09077v3)
Abstract: A crucial reason for the success of existing NeRF-based methods is to build a neural density field for the geometry representation via multiple perceptron layers (MLPs). MLPs are continuous functions, however, real geometry or density field is frequently discontinuous at the interface between the air and the surface. Such a contrary brings the problem of unfaithful geometry representation. To this end, this paper proposes spiking NeRF, which leverages spiking neurons and a hybrid Artificial Neural Network (ANN)-Spiking Neural Network (SNN) framework to build a discontinuous density field for faithful geometry representation. Specifically, we first demonstrate the reason why continuous density fields will bring inaccuracy. Then, we propose to use the spiking neurons to build a discontinuous density field. We conduct a comprehensive analysis for the problem of existing spiking neuron models and then provide the numerical relationship between the parameter of the spiking neuron and the theoretical accuracy of geometry. Based on this, we propose a bounded spiking neuron to build the discontinuous density field. Our method achieves SOTA performance. The source code and the supplementary material are available at https://github.com/liaozhanfeng/Spiking-NeRF.
- A Review of Non-Cognitive Applications for Neuromorphic Computing. Neuromorphic Computing and Engineering.
- Controlling Neural Level Sets. NIPS.
- Neural RGB-D Surface Reconstruction. In CVPR.
- Adversarial Training for Probabilistic Spiking Neural Networks. In 2018 IEEE 19th International Workshop on Signal Processing Advances in Wireless Communications (SPAWC).
- Mip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance Fields. In ICCV.
- NeRD: Neural Reflectance Decomposition From Image Collections. In ICCV.
- Improving Neural Implicit Surfaces Geometry with Patch Warping. In CVPR.
- Depth-Supervised NeRF: Fewer Views and Faster Training for Free. In CVPR.
- Deep Residual Learning in Spiking Neural Networks. NIPS.
- Geo-Neus: Geometry-Consistent Neural Implicit Surfaces Learning for Multi-view Reconstruction. NIPS.
- Event-Based Angular Velocity Regression with Spiking Networks. In ICRA.
- NeRFReN: Neural Radiance Fields with Reflections. In CVPR.
- Symmetric-Threshold ReLU for Fast and Nearly Lossless ANN-SNN Conversion. Machine Intelligence Research.
- Dex-NeRF: Using a Neural Radiance Field to Grasp Transparent Objects. In 5th Annual Conference on Robot Learning.
- Large Scale Multi-View Stereopsis Evaluation. In CVPR.
- Hybrid SNN-ANN: Energy-Efficient Classification and Object Detection for Event-Based Vision. In DAGM German Conference on Pattern Recognition.
- Spike-FlowNet: Event-Based Optical Flow Estimation with Energy-Efficient Hybrid Neural Networks. In ECCV.
- Bio-Mimetic High-Speed Target Localization with Fused Frame and Event Vision for Edge Application. Frontiers in Neuroscience.
- Shape Reconstruction Using Volume Sweeping and Learned Photoconsistency. In ECCV.
- SeaThru-NeRF: Neural Radiance Fields in Scattering Media. In CVPR.
- A Fast and Energy-Efficient SNN Processor With Adaptive Clock/Event-Driven Computation Scheme and Online Learning. IEEE Transactions on Circuits and Systems I: Regular Papers.
- Brain-Inspired Multilayer Perceptron with Spiking Neurons. In CVPR.
- A Free Lunch from ANN: Towards Efficient, Accurate Spiking Neural Networks Calibration. In ICML.
- Differentiable Spike: Rethinking Gradient-Descent for Training Spiking Neural Networks. NIPS.
- Enhancing Spiking Neural Networks with Hybrid Top-Down Attention. Frontiers in Neuroscience.
- Neural Sparse Voxel Fields. NIPS.
- DIST: Rendering Deep Implicit Signed Distance Function With Differentiable Sphere Tracing. In CVPR.
- Learning Nonlinear Operators via DeepONet Based on the Universal Approximation Theorem of Operators. Nature Machine Intelligence.
- Is Spiking Secure? A Comparative Study on the Security Vulnerabilities of Spiking and Deep Neural Networks. In IJCNN.
- NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis. Communications of the ACM.
- Surrogate Gradient Learning in Spiking Neural Networks: Bringing the Power of Gradient-Based Optimization to Spiking Neural Networks. IEEE Signal Processing Magazine.
- Differentiable Volumetric Rendering: Learning Implicit 3D Representations Without 3D Supervision. In CVPR.
- UNISURF: Unifying Neural Implicit Surfaces and Radiance Fields for Multi-View Reconstruction. In ICCV.
- Ororbia, A. 2023. Spiking Neural Predictive Coding for Continually Learning from Data Streams. Neurocomputing.
- Convolutional Occupancy Networks. In ECCV.
- Neural Body: Implicit Neural Representations With Structured Latent Codes for Novel View Synthesis of Dynamic Humans. In CVPR.
- Deep Learning with Spiking Neurons: Opportunities and Challenges. Frontiers in Neuroscience.
- D-NeRF: Neural Radiance Fields for Dynamic Scenes. In CVPR.
- StereoSpike: Depth Learning With a Spiking Neural Network. IEEE Access.
- Spiking PointNet: Spiking Neural Networks for Point Clouds. In NIPS.
- PIFu: Pixel-Aligned Implicit Function for High-Resolution Clothed Human Digitization. In ICCV.
- GRAF: Generative Radiance Fields for 3D-Aware Image Synthesis. NIPS.
- Inherent Adversarial Robustness of Deep Spiking Neural Networks: Effects of Discrete Input Encoding and Non-linear Activations. In ECCV.
- SLAYER: Spike Layer Error Reassignment in Time. NIPS.
- Scene Representation Networks: Continuous 3D-Structure-Aware Neural Scene Representations. NIPS.
- Ref-NeRF: Structured View-Dependent Appearance for Neural Radiance Fields. In CVPR.
- NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstruction. In NIPS.
- HF-NeuS: Improved Surface Reconstruction Using High-Frequency Details. NIPS.
- PET-NeuS: Positional Encoding Tri-Planes for Neural Surfaces. In CVPR.
- DISN: Deep Implicit Surface Network for High-quality Single-view 3D Reconstruction. NIPS.
- Volume Rendering of Neural Implicit Surfaces. NIPS.
- Multiview Neural Surface Reconstruction by Disentangling Geometry and Appearance. NIPS.
- Defects Recognition of Train Wheelset Tread Based on Improved Spiking Neural Network. Chinese Journal of Electronics.
- Spiking Transformers for Event-Based Single Object Tracking. In CVPR.
- Predicting the Temporal-Dynamic Trajectories of Cortical Neuronal Responses in Non-Human Primates Based on Deep Spiking Neural Network. Cognitive Neurodynamics.
- NeRF++: Analyzing and Improving Neural Radiance Fields. arXiv preprint arXiv:2010.07492.
- NeRFactor: Neural Factorization of Shape and Reflectance Under an Unknown Illumination. TOG.
- A Framework for the General Design and Computation of Hybrid Neural Networks. Nature Communications.
- Event-Based Video Reconstruction via Potential-Assisted Spiking Neural Network. In CVPR.
- Towards a New Paradigm for Brain-Inspired Computer Vision. Machine Intelligence Research.
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
Top Community Prompts
Collections
Sign up for free to add this paper to one or more collections.