Papers
Topics
Authors
Recent
Search
2000 character limit reached

Boundary-Aware Value Function Generation for Safe Stochastic Motion Planning

Published 22 Mar 2024 in cs.RO | (2403.14956v1)

Abstract: Navigation safety is critical for many autonomous systems such as self-driving vehicles in an urban environment. It requires an explicit consideration of boundary constraints that describe the borders of any infeasible, non-navigable, or unsafe regions. We propose a principled boundary-aware safe stochastic planning framework with promising results. Our method generates a value function that can strictly distinguish the state values between free (safe) and non-navigable (boundary) spaces in the continuous state, naturally leading to a safe boundary-aware policy. At the core of our solution lies a seamless integration of finite elements and kernel-based functions, where the finite elements allow us to characterize safety-critical states' borders accurately, and the kernel-based function speeds up computation for the non-safety-critical states. The proposed method was evaluated through extensive simulations and demonstrated safe navigation behaviors in mobile navigation tasks. Additionally, we demonstrate that our approach can maneuver safely and efficiently in cluttered real-world environments using a ground vehicle with strong external disturbances, such as navigating on a slippery floor and against external human intervention.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (94)
  1. Firm: Sampling-based feedback motion-planning under motion uncertainty and imperfect measurements. The International Journal of Robotics Research, 33(2):268–304, 2014.
  2. Wind-energy based path planning for unmanned aerial vehicles using markov decision processes. In 2013 IEEE International Conference on Robotics and Automation (ICRA), pages 784–789. IEEE, 2013.
  3. Reachability analysis of nonlinear systems with uncertain parameters using conservative linearization. In 2008 47th IEEE Conference on Decision and Control, pages 4042–4048. IEEE, 2008.
  4. Learning near-optimal policies with bellman-residual minimization based fitted policy iteration and a single sample path. Machine Learning, 71(1):89–129, 2008.
  5. Francis Bach. On the equivalence between kernel quadrature rules and random feature expansions. The Journal of Machine Learning Research, 18(1):714–751, 2017.
  6. Optimal path planning of a target-following fixed-wing uav using sequential decision processes. In 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems, pages 2955–2962. IEEE, 2013.
  7. Hamilton-jacobi reachability: A brief overview and recent advances. In 2017 IEEE 56th Annual Conference on Decision and Control (CDC), pages 2242–2253. IEEE, 2017.
  8. Richard E Bellman. Adaptive control processes: a guided tour, volume 2045. Princeton university press, 2015.
  9. Robust model predictive control: A survey. In Robustness in identification and control, pages 207–226. Springer, 1999.
  10. Random search for hyper-parameter optimization. Journal of machine learning research, 13(2), 2012.
  11. Dimitri Bertsekas. Dynamic programming and optimal control: Volume I, Athena scientific, 2012.
  12. Dimitri P Bertsekas. Dynamic programming and suboptimal control: A survey from adp to mpc. European Journal of Control, 11(4-5):310–334, 2005.
  13. Neuro-dynamic programming, volume 5. Athena Scientific Belmont, MA, 1996.
  14. On the taylor expansion of value functions. Operations Research, 68(2):631–654, 2020.
  15. The mathematical theory of finite element methods, volume 3. Springer, 2008.
  16. Numerical analysis. Cengage learning, 2015.
  17. Real-time safe trajectory generation for quadrotor flight in cluttered environments. In 2015 IEEE International Conference on Robotics and Biomimetics (ROBIO), pages 1678–1685. IEEE, 2015.
  18. Mo Chen and Claire J Tomlin. Hamilton–jacobi reachability: Some recent theoretical advances and applications in unmanned airspace management. Annual Review of Control, Robotics, and Autonomous Systems, 1:333–358, 2018.
  19. Fast reachable set approximations via state decoupling disturbances. In 2016 IEEE 55th Conference on Decision and Control (CDC), pages 191–196. IEEE, 2016.
  20. Exact and efficient hamilton-jacobi guaranteed safety analysis via system decomposition. In 2017 IEEE International Conference on Robotics and Automation (ICRA), pages 87–92. IEEE, 2017.
  21. Delaunay mesh generation. CRC Press Boca Raton, FL, 2013.
  22. Gaussian process dynamic programming. Neurocomputing, 72(7-9):1508–1524, 2009.
  23. Efficient mixed-integer planning for uavs in cluttered environments. In 2015 IEEE international conference on robotics and automation (ICRA), pages 42–49. IEEE, 2015.
  24. Exploration-guided reward shaping for reinforcement learning under sparse rewards. Advances in Neural Information Processing Systems, 35:5829–5842, 2022.
  25. Bayes meets bellman: The gaussian process approach to temporal difference learning. In Proceedings of the 20th International Conference on Machine Learning (ICML-03), pages 154–161, 2003.
  26. Lawrence C. Evans. Partial Differential Equations: Second Edition (Graduate Series in Mathematics). American Mathematical Society, 2010.
  27. A theoretical analysis of deep q-learning. In Learning for Dynamics and Control, pages 486–489. PMLR, 2020.
  28. Sense and collision avoidance of unmanned aerial vehicles using markov decision process and flatness approach. In 2015 IEEE International Conference on Information and Automation, pages 714–719. IEEE, 2015.
  29. Architecting the finite element method pipeline for the GPU. Journal of computational and applied mathematics, 257, pages 195–211, 2014.
  30. Asymptotically optimal sampling-based motion planning methods. Annual Review of Control, Robotics, and Autonomous Systems, 4.
  31. Online quadrotor trajectory generation and autonomous navigation on point clouds. In 2016 IEEE International Symposium on Safety, Security, and Rescue Robotics (SSRR), pages 139–146. IEEE, 2016.
  32. Online safe trajectory generation for quadrotors using fast marching method and bernstein basis polynomial. In 2018 IEEE International Conference on Robotics and Automation (ICRA), pages 344–351. IEEE, 2018.
  33. Flying on point clouds: Online trajectory generation and autonomous navigation for quadrotors in cluttered environments. Journal of Field Robotics, 36(4):710–733, 2019.
  34. Algorithmic survey of parametric value function approximation. IEEE Transactions on Neural Networks and Learning Systems, 24(6):845–867, 2013.
  35. Efficient high-dimensional stochastic optimal motion control using tensor-train decomposition. In Robotics: Science and Systems, 2015.
  36. Kernel methods in machine learning. The annals of statistics, 36(3):1171–1220, 2008.
  37. Thomas JR Hughes. The finite element method: linear static and dynamic finite element analysis. Courier Corporation, 2012.
  38. An incremental sampling-based algorithm for stochastic optimal control. In The International Journal of Robotics Research, volume 35, pages 305–333. Sage Publications, 2016.
  39. Rudolf Emil Kalman. Contributions to the theory of optimal control. Bol. soc. mat. mexicana, 5(2):102–119, 1960.
  40. Optimal control as a graphical model inference problem. Machine learning, 87(2):159–182, 2012.
  41. Sampling-based algorithms for optimal motion planning. The international journal of robotics research, 30(7):846–894, 2011.
  42. Reinforcement learning in robotics: A survey. The International Journal of Robotics Research, 32(11):1238–1274, 2013.
  43. Bridging the gap between safety and real-time performance in receding-horizon trajectory design for mobile robots. The International Journal of Robotics Research, 39(12):1419–1469, 2020.
  44. Gaussian processes in reinforcement learning. In Advances in Neural Information Processing Systems, Volume 16, pages 751–758, 2004.
  45. Robust model predictive control using tubes. Automatica, 40(1):125–133, 2004.
  46. Steven M LaValle. Planning algorithms. Cambridge university press, 2006.
  47. Planning dynamically feasible trajectories for quadrotors using safe flight corridors in 3-d complex environments. IEEE Robotics and Automation Letters, 2(3):1688–1695, 2017.
  48. Towards Efficient MPPI Trajectory Generation with Unscented Guidance: U-MPPI Control Strategy. In arXiv preprint arXiv:2306.12369, 2023.
  49. Robust online motion planning with regions of finite time invariance. In Algorithmic foundations of robotics X, pages 543–558. Springer, 2013.
  50. Funnel libraries for real-time robust feedback motion planning. The International Journal of Robotics Research, 36(8):947–982, 2017.
  51. The office marathon: Robust navigation in an indoor office environment. In 2010 IEEE international conference on robotics and automation, pages 300–307. IEEE, 2010.
  52. Survey of finite element method-based real-time simulations. Applied Sciences, 9 (14), page 2775, 2019.
  53. Minimum snap trajectory generation and control for quadrotors. In 2011 IEEE international conference on robotics and automation, pages 2520–2525. IEEE, 2011.
  54. Ian M Mitchell. The flexible, extensible and efficient toolbox of level set methods. Journal of Scientific Computing, 35:300–329, 2008.
  55. Variable resolution discretization in optimal control. Machine learning, 49(2-3):291–323, 2002.
  56. Finite-time bounds for fitted value iteration. Journal of Machine Learning Research, 9(May):815–857, 2008.
  57. An introduction to the mathematical theory of finite elements. Courier Corporation, 2012.
  58. Continuous-time trajectory optimization for online uav replanning. In 2016 IEEE/RSJ international conference on intelligent robots and systems (IROS), pages 5332–5339. IEEE, 2016a.
  59. Signed distance fields: A natural representation for both mapping and planning. In RSS 2016 Workshop: Geometry and Beyond-Representations, Physics, and Scene Understanding for Robotics. University of Michigan, 2016b.
  60. Kernel-based reinforcement learning. Machine learning, 49(2-3):161, 2002.
  61. Any-time path-planning: Time-varying wind field+ moving obstacles. In 2016 IEEE International Conference on Robotics and Automation (ICRA), pages 2575–2582. IEEE, 2016.
  62. Reinforcement learning with function-valued action spaces for partial differential equation control. arXiv preprint arXiv:1806.06931, 2018.
  63. Challenges of machine learning applied to safety-critical cyber-physical systems. Machine Learning and Knowledge Extraction, 2(4):579–602, 2020.
  64. Risk-aware path planning for autonomous underwater vehicles using predictive ocean models. Journal of Field Robotics, 30(5):741–762, 2013.
  65. Martin L Puterman. Markov decision processes: discrete stochastic dynamic programming. John Wiley & Sons, 2014.
  66. Chomp: Gradient optimization techniques for efficient motion planning. In 2009 IEEE International Conference on Robotics and Automation, pages 489–494. IEEE, 2009.
  67. Model predictive control: theory, computation, and design, volume 2. Nob Hill Publishing Madison, WI, 2017.
  68. Polynomial trajectory planning for aggressive quadrotor flight in dense indoor environments. In Robotics research, pages 649–666. Springer, 2016.
  69. Learning by playing solving sparse reward tasks from scratch. In International Conference on Machine Learning, pages 4344–4353. PMLR, 2018.
  70. Real-Time Robust Receding Horizon Planning Using Hamilton–Jacobi Reachability Analysis. In IEEE Transactions on Robotics, page 90–109, 2022.
  71. Kernel methods for pattern analysis. Cambridge university press, 2004.
  72. Stochastic extended LQR for optimization-based motion planning under uncertainty. IEEE Transactions on Automation Science and Engineering, 13(2):437–447, 2016.
  73. Reinforcement learning: An introduction. MIT press, 2018.
  74. Kernelized value function approximation for reinforcement learning. In Proceedings of the 26th Annual International Conference on Machine Learning, pages 1017–1024, 2009.
  75. Lqr-trees: Feedback motion planning via sums-of-squares verification. The International Journal of Robotics Research, 29(8):1038–1052, 2010.
  76. A generalized path integral control approach to reinforcement learning. The Journal of Machine Learning Research, 11:3137–3181, 2010.
  77. Kernel-based least squares policy iteration for reinforcement learning. IEEE Transactions on Neural Networks, 18(4):973–992, 2007.
  78. Sebastian Thrun. Probabilistic robotics. Communications of the ACM, 45(3):52–57, 2002.
  79. Probabilistic robotics, volume 1. MIT press Cambridge, 2000.
  80. Lqg-mp: Optimized path planning for robots with motion uncertainty and imperfect state information. The International Journal of Robotics Research, 30(7):895–913, 2011.
  81. Dustin J Webb and Jur van den Berg. Kinodynamic rrt*: Optimal motion planning for systems with linear differential constraints. arXiv preprint arXiv:1205.5088, 2012.
  82. Holger Wendland. Scattered data approximation, volume 17. Cambridge university press, 2004.
  83. Information theoretic mpc for model-based reinforcement learning. In 2017 IEEE International Conference on Robotics and Automation (ICRA), pages 1714–1721. IEEE, 2017.
  84. Robust sampling based model predictive control with sparse objective information. In Robotics: Science and Systems, 2018.
  85. Reachable space characterization of markov decision processes with time variability. In Proceedings of Robotics: Science and Systems, FreiburgimBreisgau, Germany, June 2019. 10.15607/RSS.2019.XV.069.
  86. Kernel Taylor-Based Value Function Approximation for Continuous-State Markov Decision Processes. In Proceedings of Robotics: Science and Systems, Corvalis, Oregon, USA, July 2020. 10.15607/RSS.2020.XVI.050.
  87. Autonomous Navigation of AGVs in Unknown Cluttered Environments: log-MPPI Control Strategy. IEEE Robotics and Automation Letters, pages 10240-10247, 2022.
  88. Causal Inference for De-biasing Motion Estimation from Robotic Observational Data. In 2023 IEEE International Conference on Robotics and Automation (ICRA), pages 3008-3014, 2023.
  89. Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. Journal of Computational physics, pages 686–707, 2019.
  90. Exact imposition of boundary conditions with distance functions in physics-informed deep neural networks. Computer Methods in Applied Mechanics and Engineering. 389 (2022): 114333.
  91. Sample-optimal parametric q-learning using linearly additive features. In International Conference on Machine Learning, pages 6995–7004. PMLR, 2019.
  92. Robust and efficient quadrotor trajectory generation for fast autonomous flight. IEEE Robotics and Automation Letters, 4(4):3529–3536, 2019.
  93. Robust real-time uav replanning using guided gradient-based optimization and topological paths. In 2020 IEEE International Conference on Robotics and Automation (ICRA), pages 1208–1214. IEEE, 2020.
  94. Value function approximation and model predictive control. In 2013 IEEE symposium on adaptive dynamic programming and reinforcement learning (ADPRL), pages 100–107. IEEE, 2023.

Summary

No one has generated a summary of this paper yet.

Sign Up to Summarize

Paper to Video (Beta)

No one has generated a video about this paper yet.

Sign Up to Generate All Videos Subscribe on YouTube

Whiteboard

No one has generated a whiteboard explanation for this paper yet.

Sign Up to Generate

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Generate Now

Continue Learning

We haven't generated follow-up questions for this paper yet.

Generate Now

Collections

Sign up for free to add this paper to one or more collections.

Sign Up