Papers
Topics
Authors
Recent
Search
2000 character limit reached

Recursively Feasible Probabilistic Safe Online Learning with Control Barrier Functions

Published 23 Aug 2022 in eess.SY, cs.LG, cs.SY, and math.OC | (2208.10733v3)

Abstract: Learning-based control has recently shown great efficacy in performing complex tasks for various applications. However, to deploy it in real systems, it is of vital importance to guarantee the system will stay safe. Control Barrier Functions (CBFs) offer mathematical tools for designing safety-preserving controllers for systems with known dynamics. In this article, we first introduce a model-uncertainty-aware reformulation of CBF-based safety-critical controllers using Gaussian Process (GP) regression to close the gap between an approximate mathematical model and the real system, which results in a second-order cone program (SOCP)-based control design. We then present the pointwise feasibility conditions of the resulting safety controller, highlighting the level of richness that the available system information must meet to ensure safety. We use these conditions to devise an event-triggered online data collection strategy that ensures the recursive feasibility of the learned safety controller. Our method works by constantly reasoning about whether the current information is sufficient to ensure safety or if new measurements under active safe exploration are required to reduce the uncertainty. As a result, our proposed framework can guarantee the forward invariance of the safe set defined by the CBF with high probability, even if it contains a priori unexplored regions. We validate the proposed framework in two numerical simulation experiments.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (45)
  1. A. D. Ames, X. Xu, J. W. Grizzle, and P. Tabuada, “Control barrier function based quadratic programs for safety critical systems,” IEEE Transactions on Automatic Control, vol. 62, pp. 3861–3876, 2017.
  2. S. Bansal, M. Chen, S. Herbert, and C. J. Tomlin, “Hamilton-jacobi reachability: A brief overview and recent advances,” in 2017 56th IEEE Conference on Decision and Control (CDC), 2017.
  3. K. P. Wabersich and M. N. Zeilinger, “A predictive safety filter for learning-based control of constrained nonlinear dynamical systems,” Automatica, vol. 129, p. 109597, 2021.
  4. Q. Nguyen and K. Sreenath, “L1 adaptive control for bipedal robots with control lyapunov function based quadratic programs,” in American Control Conference, Chicago, IL, July 2015, pp. 862–867.
  5. A. J. Taylor and A. D. Ames, “Adaptive safety with control barrier functions,” in American Control Conference, 2020, pp. 1399–1405.
  6. B. T. Lopez, J.-J. E. Slotine, and J. P. How, “Robust adaptive control barrier functions: An adaptive and data-driven approach to safety,” IEEE Control Systems Letters, vol. 5, no. 3, pp. 1031–1036, 2020.
  7. M. Black and D. Panagou, “Adaptation for validation of a consolidated control barrier function based control synthesis,” arXiv preprint arXiv:2209.08170, 2022.
  8. Q. Nguyen and K. Sreenath, “Robust safety-critical control for dynamic robotics,” IEEE Transactions on Automatic Control, 2021.
  9. J. J. Choi, D. Lee, K. Sreenath, C. J. Tomlin, and S. L. Herbert, “Robust control barrier–value functions for safety-critical control,” in 2021 60th IEEE Conference on Decision and Control (CDC).   IEEE, 2021, pp. 6814–6821.
  10. S. Kolathaya and A. D. Ames, “Input-to-state safety with control barrier functions,” IEEE control systems letters, vol. 3, no. 1, pp. 108–113, 2018.
  11. A. Alan, A. J. Taylor, C. R. He, A. D. Ames, and G. Orosz, “Control barrier functions and input-to-state safety with application to automated vehicles,” IEEE Transactions on Control Systems Technology, 2023.
  12. M. Krstic, “Inverse optimal safety filters,” arXiv preprint arXiv:2112.08225, 2021.
  13. A. J. Taylor, V. D. Dorobantu, H. M. Le, Y. Yue, and A. D. Ames, “Episodic learning with control lyapunov functions for uncertain robotic systems,” in IEEE/RSJ International Conference on Intelligent Robots and Systems, 2019, pp. 6878–6884.
  14. A. J. Taylor, A. Singletary, Y. Yue, and A. Ames, “Learning for safety-critical control with control barrier functions,” in Learning for Dynamics and Control, 2020, pp. 708–717.
  15. T. Westenbroek, F. Castañeda, A. Agrawal, S. S. Sastry, and K. Sreenath, “Learning min-norm stabilizing control laws for systems with unknown dynamics,” in IEEE Conference on Decision and Control, 2020, pp. 737–744.
  16. J. Choi, F. Castañeda, C. Tomlin, and K. Sreenath, “Reinforcement Learning for Safety-Critical Control under Model Uncertainty, using Control Lyapunov Functions and Control Barrier Functions,” in Robotics: Science and Systems, Corvalis, OR, 2020.
  17. F. Berkenkamp, R. Moriconi, A. P. Schoellig, and A. Krause, “Safe learning of regions of attraction for uncertain, nonlinear systems with gaussian processes,” in IEEE Conference on Decision and Control, 2016, pp. 4661–4666.
  18. F. Berkenkamp, M. Turchetta, A. Schoellig, and A. Krause, “Safe model-based reinforcement learning with stability guarantees,” in Advances in Neural Information Processing Systems.   Curran Associates, Inc., 2017, vol. 30, pp. 908–918.
  19. J. F. Fisac, A. K. Akametalu, M. N. Zeilinger, S. Kaynama, J. Gillula, and C. J. Tomlin, “A general safety framework for learning-based control in uncertain robotic systems,” IEEE Transactions on Automatic Control, vol. 64, no. 7, pp. 2737–2752, 2018.
  20. J. Umlauft, L. Pöhler, and S. Hirche, “An uncertainty-based control lyapunov approach for control-affine systems modeled by gaussian process,” IEEE Control Systems Letters, vol. 2, pp. 483–488, 2018.
  21. D. D. Fan, J. Nguyen, R. Thakker, N. Alatur, A. a. Agha-mohammadi, and E. A. Theodorou, “Bayesian learning-based adaptive control for safety critical systems,” in IEEE International Conference on Robotics and Automation, 2020, pp. 4093–4099.
  22. R. Cheng, M. J. Khojasteh, A. D. Ames, and J. W. Burdick, “Safe multi-agent interaction through robust control barrier functions with learned uncertainties,” in IEEE Conference on Decision and Control, 2020, pp. 777–783.
  23. M. H. Cohen and C. Belta, “Safe exploration in model-based reinforcement learning using control barrier functions,” arXiv preprint arXiv:2104.08171, 2021.
  24. F. Castañeda, J. J. Choi, B. Zhang, C. J. Tomlin, and K. Sreenath, “Gaussian process-based min-norm stabilizing controller for control-affine systems with uncertain input effects and dynamics,” in American Control Conference, 2021.
  25. F. Castañeda, J. J. Choi, B. Zhang, C. J. Tomlin, and K. Sreenath, “Pointwise feasibility of gaussian process-based safety-critical control under model uncertainty,” in IEEE Conference on Decision and Control, 2021, pp. 6762–6769.
  26. A. J. Taylor, V. D. Dorobantu, S. Dean, B. Recht, Y. Yue, and A. D. Ames, “Towards robust data-driven control synthesis for nonlinear systems with actuation uncertainty,” in IEEE Conference on Decision and Control, 2021, pp. 6469–6476.
  27. M. Greeff, A. W. Hall, and A. P. Schoellig, “Learning a stability filter for uncertain differentially flat systems using gaussian processes,” in IEEE Conference on Decision and Control, 2021, pp. 789–794.
  28. V. Dhiman, M. J. Khojasteh, M. Franceschetti, and N. Atanasov, “Control barriers in bayesian learning of system dynamics,” IEEE Transactions on Automatic Control, 2021.
  29. L. Brunke, S. Zhou, and A. P. Schoellig, “Barrier bayesian linear regression: Online learning of control barrier conditions for safety-critical control of uncertain systems,” in Learning for Dynamics and Control, 2022, pp. 881–892.
  30. J. Umlauft and S. Hirche, “Feedback linearization based on gaussian processes with event-triggered online learning,” IEEE Transactions on Automatic Control, vol. 65, no. 10, pp. 4154–4169, 2019.
  31. X. Xu, P. Tabuada, J. W. Grizzle, and A. D. Ames, “Robustness of control barrier functions for safety critical control,” IFAC-PapersOnLine, vol. 48, no. 27, pp. 54–61, 2015.
  32. C. Dawson, Z. Qin, S. Gao, and C. Fan, “Safe nonlinear control using robust neural lyapunov-barrier functions,” in Conference on Robot Learning.   PMLR, 2022, pp. 1724–1735.
  33. Z. Qin, D. Sun, and C. Fan, “Sablas: Learning safe control for black-box dynamical systems,” IEEE Robotics and Automation Letters, vol. 7, no. 2, pp. 1928–1935, 2022.
  34. P. Jagtap, G. J. Pappas, and M. Zamani, “Control barrier functions for unknown nonlinear systems using gaussian processes,” in 2020 59th IEEE Conference on Decision and Control (CDC).   IEEE, 2020, pp. 3699–3704.
  35. L. Lindemann, A. Robey, L. Jiang, S. Tu, and N. Matni, “Learning robust output control barrier functions from safe expert demonstrations,” arXiv preprint arXiv:2111.09971, 2021.
  36. W. Jin, Z. Wang, Z. Yang, and S. Mou, “Neural certificates for safe control policies,” arXiv preprint arXiv:2006.08465, 2020.
  37. D. Duvenaud, “Automatic model construction with gaussian processes,” Ph.D. dissertation, University of Cambridge, 2014.
  38. N. Srinivas, A. Krause, S. Kakade, and M. Seeger, “Gaussian process optimization in the bandit setting: No regret and experimental design,” in International Conference on Machine Learning, 2010.
  39. J. Umlauft, T. Beckers, A. Capone, A. Lederer, and S. Hirche, “Smart forgetting for safe online learning with gaussian processes,” in Learning for Dynamics and Control, 2020, pp. 160–169.
  40. H. Liu, Y.-S. Ong, X. Shen, and J. Cai, “When gaussian process meets big data: A review of scalable gps,” IEEE transactions on neural networks and learning systems, vol. 31, no. 11, pp. 4405–4423, 2020.
  41. S. Dean, A. Taylor, R. Cosner, B. Recht, and A. Ames, “Guaranteeing safety of learned perception modules via measurement-robust control barrier functions,” in Conference on Robot Learning, 2021.
  42. J. Buch, S.-C. Liao, and P. Seiler, “Robust control barrier functions with sector-bounded uncertainties,” IEEE Control Systems Letters, vol. 6, pp. 1994–1999, 2021.
  43. T. Lew, A. Sharma, J. Harrison, E. Schmerling, and M. Pavone, “On the problem of reformulating systems with uncertain dynamics as a stochastic differential equation,” arXiv preprint arXiv:2111.06084, 2021.
  44. G. Still, “Lectures on parametric optimization: An introduction,” Optimization Online, 2018.
  45. J. Lygeros, K. H. Johansson, S. N. Simic, J. Zhang, and S. S. Sastry, “Dynamical properties of hybrid automata,” IEEE Transactions on Automatic Control, vol. 48, no. 1, pp. 2–17, 2003.
Citations (5)
View on Semantic Scholar

Summary

No one has generated a summary of this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Sign Up to Summarize

Paper to Video (Beta)

Whiteboard

No one has generated a whiteboard explanation for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Sign Up to Generate

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now

Continue Learning

We haven't generated follow-up questions for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now

Collections

Sign up for free to add this paper to one or more collections.

User Circle Single Streamline Icon: https://streamlinehq.com Sign Up