Papers
Topics
Authors
Recent
Search
2000 character limit reached

Neural Distributed Controllers with Port-Hamiltonian Structures

Published 26 Mar 2024 in eess.SY and cs.SY | (2403.17785v1)

Abstract: Controlling large-scale cyber-physical systems necessitates optimal distributed policies, relying solely on local real-time data and limited communication with neighboring agents. However, finding optimal controllers remains challenging, even in seemingly simple scenarios. Parameterizing these policies using Neural Networks (NNs) can deliver good performance, but their sensitivity to small input changes can destabilize the closed-loop system. This paper addresses this issue for a network of nonlinear dissipative systems. Specifically, we leverage well-established port-Hamiltonian structures to characterize deep distributed control policies with closed-loop stability guarantees and a finite $\mathcal{L}_2$ gain, regardless of specific NN parameters. This eliminates the need to constrain the parameters during optimization and enables training with standard methods like stochastic gradient descent. A numerical study on the consensus control of Kuramoto oscillators demonstrates the effectiveness of the proposed controllers.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (45)
  1. H. S. Witsenhausen, “A counterexample in stochastic optimum control,” SIAM Journal on Control, vol. 6, no. 1, pp. 131–147, 1968.
  2. L. Lessard and S. Lall, “Quadratic invariance is necessary and sufficient for convexity,” in IEEE American Control Conference (ACC), 2011, pp. 5360–5362.
  3. L. Furieri, C. L. Galimberti, M. Zakwan, and G. Ferrari-Trecate, “Distributed neural network control with dependability guarantees: a compositional port-hamiltonian approach,” in Learning for Dynamics and Control Conference.   PMLR, 2022, pp. 571–583.
  4. L. Brunke, M. Greeff, A. W. Hall, Z. Yuan, S. Zhou, J. Panerati, and A. P. Schoellig, “Safe learning in robotics: From learning-based control to safe reinforcement learning,” Annual Review of Control, Robotics, and Autonomous Systems, vol. 5, pp. 411–444, 2022.
  5. H. Tsukamoto, S.-J. Chung, and J.-J. E. Slotine, “Contraction theory for nonlinear stability analysis and learning-based control: A tutorial overview,” Annual Reviews in Control, vol. 52, pp. 135–169, 2021.
  6. C. Dawson, S. Gao, and C. Fan, “Safe control with learned certificates: A survey of neural Lyapunov, barrier, and contraction methods,” arXiv preprint arXiv:2202.11762, 2022.
  7. T. X. Nghiem, J. Drgoňa, C. Jones, Z. Nagy, R. Schwan, B. Dey, A. Chakrabarty, S. Di Cairano, J. A. Paulson, A. Carron et al., “Physics-informed machine learning for modeling and control of dynamical systems,” arXiv preprint arXiv:2306.13867, 2023.
  8. G. I. Beintema, M. Schoukens, and R. Tóth, “Deep subspace encoders for nonlinear system identification,” Automatica, vol. 156, p. 111210, 2023.
  9. C. Verhoek, G. I. Beintema, S. Haesaert, M. Schoukens, and R. Tóth, “Deep-learning-based identification of LPV models for nonlinear systems,” in 2022 IEEE 61st Conference on Decision and Control (CDC).   IEEE, 2022, pp. 3274–3280.
  10. M. Revay, R. Wang, and I. R. Manchester, “Recurrent equilibrium networks: Flexible dynamic models with guaranteed stability and robustness,” IEEE Transactions on Automatic Control, 2023.
  11. R. Wang, N. H. Barbara, M. Revay, and I. R. Manchester, “Learning over all stabilizing nonlinear controllers for a partially-observed linear system,” IEEE Control Systems Letters, vol. 7, pp. 91–96, 2022.
  12. M. Scandella, M. Bin, and T. Parisini, “Kernel-based identification of incrementally input-to-state stable nonlinear systems,” IFAC-PapersOnLine, vol. 56, no. 2, pp. 5127–5132, 2023.
  13. M. Zakwan, L. Di Natale, B. Svetozarevic, P. Heer, C. N. Jones, and G. F. Trecate, “Physically consistent neural odes for learning multi-physics systems,” arXiv preprint arXiv:2211.06130, 2022.
  14. L. Di Natale, M. Zakwan, P. Heer, G. F. Trecate, and C. N. Jones, “Simba: System identification methods leveraging backpropagation,” arXiv preprint arXiv:2311.13889, 2023.
  15. L. Di Natale, M. Zakwan, B. Svetozarevic, P. Heer, G. F. Trecate, and C. N. Jones, “Stable linear subspace identification: A machine learning approach,” arXiv preprint arXiv:2311.03197, 2023.
  16. T. Asikis, L. Böttcher, and N. Antulov-Fantulin, “Neural ordinary differential equation control of dynamics on graphs,” Physical Review Research, vol. 4, no. 1, p. 013221, 2022.
  17. L. Böttcher, N. Antulov-Fantulin, and T. Asikis, “AI pontryagin or how artificial neural networks learn to control dynamical systems,” Nature communications, vol. 13, no. 1, p. 333, 2022.
  18. L. Hewing, J. Kabzan, and M. N. Zeilinger, “Cautious model predictive control using gaussian process regression,” IEEE Transactions on Control Systems Technology, vol. 28, no. 6, pp. 2736–2743, 2019.
  19. L. B. Armenio, E. Terzi, M. Farina, and R. Scattolini, “Model predictive control design for dynamical systems learned by echo state networks,” IEEE Control Systems Letters, vol. 3, no. 4, pp. 1044–1049, 2019.
  20. F. Bonassi, M. Farina, J. Xie, and R. Scattolini, “On recurrent neural networks for learning-based control: recent results and ideas for future developments,” Journal of Process Control, vol. 114, pp. 92–104, 2022.
  21. E. Terzi, F. Bonassi, M. Farina, and R. Scattolini, “Learning model predictive control with long short-term memory networks,” International Journal of Robust and Nonlinear Control, vol. 31, no. 18, pp. 8877–8896, 2021.
  22. M. Zakwan, L. Xu, and G. Ferrari-Trecate, “Robust classification using contractive hamiltonian neural odes,” IEEE Control Systems Letters, vol. 7, pp. 145–150, 2022.
  23. F. Yang and N. Matni, “Communication topology co-design in graph recurrent neural network based distributed control,” arXiv preprint arXiv:2104.13868, 2021.
  24. E. Tolstaya, F. Gama, J. Paulos, G. Pappas, V. Kumar, and A. Ribeiro, “Learning decentralized controllers for robot swarms with graph neural networks,” in Conference on robot learning.   PMLR, 2020, pp. 671–682.
  25. A. Khan, E. Tolstaya, A. Ribeiro, and V. Kumar, “Graph policy gradients for large scale robot control,” in Conference on robot learning.   PMLR, 2020, pp. 823–834.
  26. F. Gama and S. Sojoudi, “Graph neural networks for distributed linear-quadratic control,” in Learning for Dynamics and Control.   PMLR, 2021, pp. 111–124.
  27. L. Brunke, M. Greeff, A. W. Hall, Z. Yuan, S. Zhou, J. Panerati, and A. P. Schoellig, “Safe learning in robotics: From learning-based control to safe reinforcement learning,” arXiv preprint arXiv:2108.06266, 2021.
  28. R. Cheng, G. Orosz, R. M. Murray, and J. W. Burdick, “End-to-end safe reinforcement learning through barrier functions for safety-critical continuous control tasks,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 33, no. 01, 2019, pp. 3387–3395.
  29. F. Berkenkamp, M. Turchetta, A. P. Schoellig, and A. Krause, “Safe model-based reinforcement learning with stability guarantees,” Advances in Neural Information Processing Systems 30, vol. 2, pp. 909–919, 2018.
  30. S. M. Richards, F. Berkenkamp, and A. Krause, “The Lyapunov neural network: Adaptive stability certification for safe learning of dynamical systems,” in Conference on Robot Learning.   PMLR, 2018, pp. 466–476.
  31. T. Koller, F. Berkenkamp, M. Turchetta, and A. Krause, “Learning-based model predictive control for safe exploration,” in 2018 IEEE conference on decision and control (CDC).   IEEE, 2018, pp. 6059–6066.
  32. P. Pauli, J. Köhler, J. Berberich, A. Koch, and F. Allgöwer, “Offset-free setpoint tracking using neural network controllers,” in Learning for Dynamics and Control.   PMLR, 2021, pp. 992–1003.
  33. S. Abdulkhader, H. Yin, P. Falco, and D. Kragic, “Learning deep energy shaping policies for stability-guaranteed manipulation,” IEEE Robotics and Automation Letters, 2021.
  34. T. Duong and N. Atanasov, “Hamiltonian-based neural ODE networks on the SE (3) manifold for dynamics learning and control,” arXiv preprint arXiv:2106.12782, 2021.
  35. D. Martinelli, C. L. Galimberti, I. R. Manchester, L. Furieri, and G. Ferrari-Trecate, “Unconstrained parametrization of dissipative and contracting neural ordinary differential equations,” arXiv preprint arXiv:2304.02976, 2023.
  36. L. Massai, D. Saccani, L. Furieri, and G. Ferrari-Trecate, “Unconstrained learning of networked nonlinear systems via free parametrization of stable interconnected operators,” arXiv preprint arXiv:2311.13967, 2023.
  37. S. Z. Khong and A. van der Schaft, “On the converse of the passivity and small-gain theorems for input–output maps,” Automatica, vol. 97, pp. 58–63, 2018.
  38. C. L. Galimberti, L. Furieri, L. Xu, and G. Ferrari-Trecate, “Hamiltonian deep neural networks guaranteeing nonvanishing gradients by design,” IEEE Transactions on Automatic Control, vol. 68, no. 5, pp. 3155–3162, 2023.
  39. M. Zakwan, M. d’Angelo, and G. Ferrari-Trecate, “Universal approximation property of hamiltonian deep neural networks,” IEEE Control Systems Letters, 2023.
  40. R. T. Q. Chen, Y. Rubanova, J. Bettencourt, and D. K. Duvenaud, “Neural ordinary differential equations,” in Advances in Neural Information Processing Systems, S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Garnett, Eds., vol. 31.   Curran Associates, Inc., 2018.
  41. F. Dörfler and F. Bullo, “Synchronization in complex networks of phase oscillators: A survey,” Automatica, vol. 50, no. 6, pp. 1539–1564, 2014.
  42. J. Wu and X. Li, “Collective synchronization of kuramoto-oscillator networks,” IEEE Circuits and Systems Magazine, vol. 20, no. 3, pp. 46–67, 2020.
  43. N. Chopra and M. W. Spong, “Passivity-based control of multi-agent systems,” Advances in robot control: from everyday physics to human-like movements, pp. 107–134, 2006.
  44. E. Haber and L. Ruthotto, “Stable architectures for deep neural networks,” Inverse problems, vol. 34, no. 1, p. 014004, 2017.
  45. D. P. Kingma and J. L. Ba, “Adam: A method for stochastic gradient descent,” in ICLR: International Conference on Learning Representations, 2015, pp. 1–15.
Citations (2)
View on Semantic Scholar

Summary

No one has generated a summary of this paper yet.

Sign Up to Summarize

Paper to Video (Beta)

No one has generated a video about this paper yet.

Sign Up to Generate All Videos Subscribe on YouTube

Whiteboard

No one has generated a whiteboard explanation for this paper yet.

Sign Up to Generate

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Generate Now

Continue Learning

We haven't generated follow-up questions for this paper yet.

Generate Now

Collections

Sign up for free to add this paper to one or more collections.

Sign Up