Papers
Topics
Authors
Recent
Search
2000 character limit reached

Efficient Discrete Physics-informed Neural Networks for Addressing Evolutionary Partial Differential Equations

Published 22 Dec 2023 in cs.NE | (2312.14608v1)

Abstract: Physics-informed neural networks (PINNs) have shown promising potential for solving partial differential equations (PDEs) using deep learning. However, PINNs face training difficulties for evolutionary PDEs, particularly for dynamical systems whose solutions exhibit multi-scale or turbulent behavior over time. The reason is that PINNs may violate the temporal causality property since all the temporal features in the PINNs loss are trained simultaneously. This paper proposes to use implicit time differencing schemes to enforce temporal causality, and use transfer learning to sequentially update the PINNs in space as surrogates for PDE solutions in different time frames. The evolving PINNs are better able to capture the varying complexities of the evolutionary equations, while only requiring minor updates between adjacent time frames. Our method is theoretically proven to be convergent if the time step is small and each PINN in different time frames is well-trained. In addition, we provide state-of-the-art (SOTA) numerical results for a variety of benchmarks for which existing PINNs formulations may fail or be inefficient. We demonstrate that the proposed method improves the accuracy of PINNs approximation for evolutionary PDEs and improves efficiency by a factor of 4-40x.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (42)
  1. Handbook of differential equations: evolutionary equations. Elsevier, 2008.
  2. Physics informed deep learning (part i): Data-driven solutions of nonlinear partial differential equations. arXiv preprint arXiv:1711.10561, 2017.
  3. Solving high-dimensional partial differential equations using deep learning. Proceedings of the National Academy of Sciences, 115(34):8505–8510, 2018.
  4. Solving parametric PDE problems with artificial neural networks. European Journal of Applied Mathematics, 32(3):421–435, 2021.
  5. The deep Ritz method: A deep learning-based numerical algorithm for solving variational problems. Communications in Mathematics and Statistics, 6(1):1–12, 2018.
  6. DGM: A deep learning algorithm for solving partial differential equations. Journal of computational physics, 375:1339–1364, 2018.
  7. PDE-Net: Learning PDEs from data. In International Conference on Machine Learning, pages 3208–3216. PMLR, 2018.
  8. Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. Journal of Computational physics, 378:686–707, 2019.
  9. DeepXDE: A deep learning library for solving differential equations. SIAM review, 63(1):208–228, 2021.
  10. Estimates on the generalization error of physics-informed neural networks for approximating PDEs. IMA Journal of Numerical Analysis, 43(1):1–43, 01 2023.
  11. Understanding and mitigating gradient flow pathologies in physics-informed neural networks. SIAM Journal on Scientific Computing, 43(5):A3055–A3081, 2021.
  12. When and why PINNs fail to train: A neural tangent kernel perspective. Journal of Computational Physics, 449:110768, 2022.
  13. Respecting causality is all you need for training physics-informed neural networks. arXiv preprint arXiv:2203.07404, 2022.
  14. Characterizing possible failure modes in physics-informed neural networks. Advances in Neural Information Processing Systems, 34:26548–26560, 2021.
  15. Conservative physics-informed neural networks on discrete domains for conservation laws: Applications to forward and inverse problems. Computer Methods in Applied Mechanics and Engineering, 365:113028, 2020.
  16. hp-VPINNs: Variational physics-informed neural networks with domain decomposition. Computer Methods in Applied Mechanics and Engineering, 374:113547, 2021.
  17. Extended Physics-informed Neural Networks (XPINNs): A generalized space-time domain decomposition based deep learning framework for nonlinear partial differential equations. In AAAI Spring Symposium: MLPS, pages 2002–2041, 2021.
  18. On the eigenvector bias of Fourier feature networks: From regression to solving multi-scale PDEs with physics-informed neural networks. Computer Methods in Applied Mechanics and Engineering, 384:113938, 2021.
  19. Transfer learning enhanced physics informed neural network for phase-field modeling of fracture. Theoretical and Applied Fracture Mechanics, 106:102447, 2020.
  20. A novel meta-learning initialization method for physics-informed neural networks. Neural Computing and Applications, 34(17):14511–14534, 2022.
  21. Uri M Ascher. Numerical methods for evolutionary differential equations. SIAM, 2008.
  22. Learning without data: Physics-informed neural networks for fast time-domain simulation. In 2021 IEEE International Conference on Communications, Control, and Computing Technologies for Smart Grids (SmartGridComm), pages 438–443. IEEE, 2021.
  23. DAE-PINN: A physics-informed neural network model for simulating differential algebraic equations with application to power networks. Neural Computing and Applications, 35(5):3789–3804, 2023.
  24. Solving Allen-Cahn and Cahn-Hilliard equations using the adaptive physics informed neural networks. Communications in Computational Physics, 29(3):930–954, 2021.
  25. A novel sequential method to train physics-informed neural networks for Allen Cahn and Cahn Hilliard equations. Computer Methods in Applied Mechanics and Engineering, 390:114474, 2022.
  26. Self-adaptive physics-informed neural networks. Journal of Computational Physics, 474:111722, 2023.
  27. A unified scalable framework for causal sweeping strategies for Physics-Informed Neural Networks (PINNs) and their temporal decompositions. arXiv preprint arXiv:2302.14227, 2023.
  28. Deep transfer operator learning for partial differential equations under conditional shift. Nature Machine Intelligence, pages 1–10, 2022.
  29. Transfer learning for deep neural network-based partial differential equations solving. Advances in Aerodynamics, 3(1):1–14, 2021.
  30. Transfer learning based physics-informed neural networks for solving inverse problems in engineering structures under different loading scenarios. Computer Methods in Applied Mechanics and Engineering, 405:115852, 2023.
  31. One-shot transfer learning of physics-informed neural networks. In ICML 2022 2nd AI for Science Workshop, 2022.
  32. Transfer learning on multifidelity data. Journal of Machine Learning for Modeling and Computing, 3(1), 2022.
  33. Transfer learning enhanced DeepONet for long-time prediction of evolution equations. arXiv preprint arXiv:2212.04663, 2022.
  34. Physics-informed neural networks with hard constraints for inverse design. SIAM Journal on Scientific Computing, 43(6):B1105–B1132, 2021.
  35. A unified hard-constraint framework for solving geometrically complex PDEs. Advances in Neural Information Processing Systems, 35:20287–20299, 2022.
  36. N Sukumar and Ankit Srivastava. Exact imposition of boundary conditions with distance functions in physics-informed deep neural networks. Computer Methods in Applied Mechanics and Engineering, 389:114333, 2022.
  37. Lawrence C Evans. Partial differential equations, volume 19. American Mathematical Society, 2022.
  38. JAX: composable transformations of Python+NumPy programs, 2018.
  39. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.
  40. Maziar Raissi. Deep hidden physics models: Deep learning of nonlinear partial differential equations. The Journal of Machine Learning Research, 19(1):932–955, 2018.
  41. John Butcher. Runge-Kutta methods. Scholarpedia, 2(9):3147, 2007.
  42. Fourier features let networks learn high frequency functions in low dimensional domains. Advances in Neural Information Processing Systems, 33:7537–7547, 2020.
Citations (1)
View on Semantic Scholar

Summary

No one has generated a summary of this paper yet.

Sign Up to Summarize

Paper to Video (Beta)

No one has generated a video about this paper yet.

Sign Up to Generate All Videos Subscribe on YouTube

Whiteboard

No one has generated a whiteboard explanation for this paper yet.

Sign Up to Generate

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Generate Now

Continue Learning

We haven't generated follow-up questions for this paper yet.

Generate Now

Authors (3)

  1. Siqi Chen 
  2. Bin Shan 
  3. Ye Li 

Collections

Sign up for free to add this paper to one or more collections.

Sign Up