Papers
Topics
Authors
Recent
Search
2000 character limit reached

Structure-Preserving Physics-Informed Neural Networks With Energy or Lyapunov Structure

Published 10 Jan 2024 in cs.LG | (2401.04986v1)

Abstract: Recently, there has been growing interest in using physics-informed neural networks (PINNs) to solve differential equations. However, the preservation of structure, such as energy and stability, in a suitable manner has yet to be established. This limitation could be a potential reason why the learning process for PINNs is not always efficient and the numerical results may suggest nonphysical behavior. Besides, there is little research on their applications on downstream tasks. To address these issues, we propose structure-preserving PINNs to improve their performance and broaden their applications for downstream tasks. Firstly, by leveraging prior knowledge about the physical system, a structure-preserving loss function is designed to assist the PINN in learning the underlying structure. Secondly, a framework that utilizes structure-preserving PINN for robust image recognition is proposed. Here, preserving the Lyapunov structure of the underlying system ensures the stability of the system. Experimental results demonstrate that the proposed method improves the numerical accuracy of PINNs for partial differential equations. Furthermore, the robustness of the model against adversarial perturbations in image data is enhanced.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (51)
  1. Ground state structures in ordered binary alloys with second neighbor interactions. Acta Metallurgica, 20(3):423–433, 1972.
  2. Input convex neural networks. In International Conference on Machine Learning, pages 146–155. PMLR, 2017.
  3. Deep equilibrium models. Advances in Neural Information Processing Systems, 32, 2019.
  4. Neural ordinary differential equations. Advances in neural information processing systems, 31, 2018.
  5. Fernando Chirigati. Inverse problem via a bayesian approach. Nature Computational Science, 1(5):304–304, 2021.
  6. Learning robust deep equilibrium models. arXiv preprint arXiv:2304.12707, 2023.
  7. Improving neural ordinary differential equations via knowledge distillation. IET Computer Vision, pages 1–11, 2023.
  8. Scientific machine learning through physics–informed neural networks: Where we are and what’s next. Journal of Scientific Computing, 92(3):88, 2022.
  9. Robust physical-world attacks on deep learning visual classification. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 1625–1634, 2018.
  10. Discrete variational derivative method: a structure-preserving numerical method for partial differential equations. CRC Press, 2010.
  11. Review on computational methods for lyapunov functions. Discrete & Continuous Dynamical Systems-B, 20(8):2291, 2015.
  12. On behavior of signs for the heat equation and a diffusion method for data separation. Communications on Pure & Applied Analysis, 12(5), 2013.
  13. Hamiltonian neural networks. Advances in neural information processing systems, 32, 2019.
  14. Structure-preserving algorithms for ordinary differential equations. Geometric numerical integration, 31, 2006.
  15. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016.
  16. Revealing hidden dynamics from time-series data by odenet. Journal of Computational Physics, 461:111203, 2022.
  17. Adversarial examples are not bugs, they are features. Advances in neural information processing systems, 32, 2019.
  18. Extended physics-informed neural networks (xpinns): A generalized space-time domain decomposition based deep learning framework for nonlinear partial differential equations. In AAAI spring symposium: MLPS, volume 10, 2021.
  19. Conservative physics-informed neural networks on discrete domains for conservation laws: Applications to forward and inverse problems. Computer Methods in Applied Mechanics and Engineering, 365:113028, 2020.
  20. Stable neural ode with lyapunov-stable equilibrium points for defending against adversarial attacks. Advances in Neural Information Processing Systems, 34:14925–14937, 2021.
  21. Physics-informed machine learning. Nature Reviews Physics, 3(6):422–440, 2021.
  22. Partial differential equation-regularized neural networks: An application to image classification, 2023.
  23. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.
  24. Learning stable deep dynamics models. Advances in neural information processing systems, 32, 2019.
  25. Characterizing possible failure modes in physics-informed neural networks. Advances in Neural Information Processing Systems, 34:26548–26560, 2021.
  26. Learning multiple layers of features from tiny images. 2009.
  27. Adversarial examples in the physical world. In Artificial intelligence safety and security, pages 99–112. Chapman and Hall/CRC, 2018.
  28. Artificial neural networks for solving ordinary and partial differential equations. IEEE transactions on neural networks, 9(5):987–1000, 1998.
  29. Interpolated adversarial training: Achieving robust neural networks without sacrificing too much accuracy. Neural Networks, 154:218–233, 2022.
  30. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11):2278–2324, 1998.
  31. Neural algorithm for solving differential equations. Journal of Computational Physics, 91(1):110–131, 1990.
  32. On the limited memory bfgs method for large scale optimization. Mathematical programming, 45(1-3):503–528, 1989.
  33. Adaptive transfer learning for pinn. Journal of Computational Physics, page 112291, 2023.
  34. Deepxde: A deep learning library for solving differential equations. SIAM review, 63(1):208–228, 2021.
  35. Towards transferable 3d adversarial attack. In ACM Multimedia Asia, pages 1–5. 2021.
  36. Deep lagrangian networks: Using physics as model prior for deep learning. In 7th International Conference on Learning Representations (ICLR). ICLR, May 2019.
  37. Towards deep learning models resistant to adversarial attacks. arXiv preprint arXiv:1706.06083, 2017.
  38. FINDE: Neural differential equations for finding and preserving invariant quantities. In The Eleventh International Conference on Learning Representations, 2023.
  39. Deep energy-based modeling of discrete-time physics. Advances in Neural Information Processing Systems, 33:13100–13111, 2020.
  40. A novel sequential method to train physics informed neural networks for allen cahn and cahn hilliard equations. Computer Methods in Applied Mechanics and Engineering, 390:114474, 2022.
  41. Solution of nonlinear ordinary differential equations by feedforward neural networks. Mathematical and Computer Modelling, 20(9):19–44, 1994.
  42. Distillation as a defense to adversarial perturbations against deep neural networks. In 2016 IEEE symposium on security and privacy (SP), pages 582–597. IEEE, 2016.
  43. Automatic differentiation in pytorch. 2017.
  44. Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. Journal of Computational physics, 378:686–707, 2019.
  45. A review of structure-preserving numerical methods for engineering applications. Computer Methods in Applied Mechanics and Engineering, 366:113067, 2020.
  46. Dissipative hamiltonian neural networks: Learning dissipative and conservative dynamics separately. arXiv preprint arXiv:2201.10085, 2022.
  47. Intriguing properties of neural networks. arXiv preprint arXiv:1312.6199, 2013.
  48. Nas-pinn: Neural architecture search-guided physics-informed neural network for solving pdes. Journal of Computational Physics, 496:112603, 2024.
  49. Recent developments in structure-preserving algorithms for oscillatory differential equations. Springer, 2018.
  50. Netzer Yuval. Reading digits in natural images with unsupervised feature learning. In Proceedings of the NIPS Workshop on Deep Learning and Unsupervised Feature Learning, 2011.
  51. Adversarial examples: Opportunities and challenges. IEEE transactions on neural networks and learning systems, 31(7):2578–2593, 2019.

Summary

No one has generated a summary of this paper yet.

Sign Up to Summarize

Paper to Video (Beta)

No one has generated a video about this paper yet.

Sign Up to Generate All Videos Subscribe on YouTube

Whiteboard

No one has generated a whiteboard explanation for this paper yet.

Sign Up to Generate

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Generate Now

Continue Learning

We haven't generated follow-up questions for this paper yet.

Generate Now

Collections

Sign up for free to add this paper to one or more collections.

Sign Up

Tweets

Sign up for free to view the 1 tweet with 1 like about this paper.

Sign Up for Free