Robust Online Convex Optimization for Disturbance Rejection
Abstract: Online convex optimization (OCO) is a powerful tool for learning sequential data, making it ideal for high precision control applications where the disturbances are arbitrary and unknown in advance. However, the ability of OCO-based controllers to accurately learn the disturbance while maintaining closed-loop stability relies on having an accurate model of the plant. This paper studies the performance of OCO-based controllers for linear time-invariant (LTI) systems subject to disturbance and model uncertainty. The model uncertainty can cause the closed-loop to become unstable. We provide a sufficient condition for robust stability based on the small gain theorem. This condition is easily incorporated as an on-line constraint in the OCO controller. Finally, we verify via numerical simulations that imposing the robust stability condition on the OCO controller ensures closed-loop stability.
- O. Anava, E. Hazan, and S. Mannor, “Online convex optimization against adversaries with memory and application to statistical arbitrage,” 2014.
- E. Hazan, “The Convex Optimization Approach to Regret Minimization,” in Optimization for Machine Learning, The MIT Press, 09 2011.
- M. Zinkevich, “Online convex programming and generalized infinitesimal gradient ascent,” in Proceedings of the 20th international conference on machine learning (icml-03), pp. 928–936, 2003.
- E. Hazan, A. Agarwal, and S. Kale, “Logarithmic regret algorithms for online convex optimization,” Machine Learning, vol. 69, no. 2-3, pp. 169–192, 2007.
- S. Shalev-Shwartz, “Online learning and online convex optimization,” Foundations and trends in Machine Learning, vol. 4, no. 2, pp. 107–194, 2011.
- E. Hazan, “Introduction to online convex optimization,” Foundations and Trends® in Optimization, vol. 2, no. 3-4, pp. 157–325, 2016.
- N. Agarwal, E. Hazan, and K. Singh, “Logarithmic regret for online control,” in Advances in Neural Information Processing Systems, pp. 10175–10184, 2019.
- D. Foster and M. Simchowitz, “Logarithmic regret for adversarial online control,” in International Conference on Machine Learning, pp. 3211–3221, 2020.
- G. Goel, N. Agarwal, K. Singh, and E. Hazan, “Best of both worlds in online control: Competitive ratio and policy regret,” arXiv preprint arXiv:2211.11219, 2022.
- N. Agarwal, B. Bullins, E. Hazan, S. Kakade, and K. Singh, “Online control with adversarial disturbances,” in International Conference on Machine Learning, pp. 111–119, PMLR, 2019.
- Pearson, 1995.
- S. Skogestad and I. Postlethwaite, Multivariable Feedback Control: Analysis and Design. John Wiley and Sons, 2nd ed., 2005.
- Y. Rahman, A. Xie, J. B. Hoagg, and D. S. Bernstein, “A tutorial and overview of retrospective cost adaptive control,” in 2016 American Control Conference, pp. 3386–3409, 2016.
- R. Venugopal and D. S. Bernstein, “Adaptive disturbance rejection using ARMARKOV/Toeplitz models,” IEEE Transactions on Control Systems Technology, vol. 8, no. 2, pp. 257–269, 2000.
- M. A. Santillo and D. S. Bernstein, “Adaptive control based on retrospective cost optimization,” Journal of guidance, control, and dynamics, vol. 33, no. 2, pp. 289–304, 2010.
- G. Goel and B. Hassibi, “Measurement-feedback control with optimal data-dependent regret,” arXiv preprint arXiv:2209.06425, 2022.
- G. Goel and B. Hassibi, “Regret-optimal estimation and control,” arXiv preprint arXiv:2106.12097, 2021.
- G. Goel and B. Hassibi, “Regret-optimal measurement-feedback control,” in Learning for Dynamics and Control, pp. 1270–1280, PMLR, 2021.
- G. Goel and B. Hassibi, “Regret-optimal control in dynamic environments,” arXiv preprint arXiv:2010.10473, 2020.
- J. Doyle, “Guaranteed margins for LQG regulators,” IEEE Transactions on Automatic Control, vol. 23, no. 4, pp. 756–757, 1978.
- H. K. Khalil, Nonlinear Systems. Upper Saddle River, NJ: Prentice-Hall, third ed. ed., 2002.
- A. Packard and J. Doyle, “The complex structured singular value,” Automatica, vol. 29, no. 1, pp. 71–109, 1993.
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
Top Community Prompts
Collections
Sign up for free to add this paper to one or more collections.