Papers
Topics
Authors
Recent
Search
2000 character limit reached

SPOT: Point Cloud Based Stereo Visual Place Recognition for Similar and Opposing Viewpoints

Published 18 Apr 2024 in cs.RO and cs.CV | (2404.12339v1)

Abstract: Recognizing places from an opposing viewpoint during a return trip is a common experience for human drivers. However, the analogous robotics capability, visual place recognition (VPR) with limited field of view cameras under 180 degree rotations, has proven to be challenging to achieve. To address this problem, this paper presents Same Place Opposing Trajectory (SPOT), a technique for opposing viewpoint VPR that relies exclusively on structure estimated through stereo visual odometry (VO). The method extends recent advances in lidar descriptors and utilizes a novel double (similar and opposing) distance matrix sequence matching method. We evaluate SPOT on a publicly available dataset with 6.7-7.6 km routes driven in similar and opposing directions under various lighting conditions. The proposed algorithm demonstrates remarkable improvement over the state-of-the-art, achieving up to 91.7% recall at 100% precision in opposing viewpoint cases, while requiring less storage than all baselines tested and running faster than all but one. Moreover, the proposed method assumes no a priori knowledge of whether the viewpoint is similar or opposing, and also demonstrates competitive performance in similar viewpoint cases.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (34)
  1. Project webpage: https://umautobots.github.io/spot.
  2. C. Campos, R. Elvira, J. J. G. Rodríguez, J. M. M. Montiel, and J. D. Tardós, “ORB-SLAM3: An accurate open-source library for visual, visual–inertial, and multimap SLAM,” IEEE Transactions on Robotics, vol. 37, no. 6, pp. 1874–1890, Dec. 2021.
  3. S. Lowry, N. Sünderhauf, P. Newman, J. J. Leonard, D. Cox, P. Corke, and M. J. Milford, “Visual place recognition: A survey,” IEEE Transactions on Robotics, vol. 32, no. 1, pp. 1–19, Feb. 2016.
  4. X. Zhang, L. Wang, and Y. Su, “Visual place recognition: A survey from deep learning perspective,” Pattern Recognition, vol. 113, p. 107760, May 2021.
  5. R. Arandjelovic, P. Gronat, A. Torii, T. Pajdla, and J. Sivic, “NetVLAD: CNN architecture for weakly supervised place recognition,” in 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA, June 2016, pp. 5297–5307.
  6. A. Ali-Bey, B. Chaib-Draa, and P. Giguére, “MixVPR: Feature mixing for visual place recognition,” in 2023 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), Waikoloa, HI, USA, Jan. 2023, pp. 2997–3006.
  7. N. Keetha, A. Mishra, J. Karhade, K. M. Jatavallabhula, S. Scherer, M. Krishna, and S. Garg, “AnyLoc: Towards universal visual place recognition,” IEEE Robotics and Automation Letters, vol. 9, no. 2, pp. 1286–1293, Feb. 2024.
  8. A. Gawel, C. D. Don, R. Siegwart, J. Nieto, and C. Cadena, “X-View: Graph-based semantic multi-view localization,” IEEE Robotics and Automation Letters, vol. 3, no. 3, pp. 1687–1694, July 2018.
  9. S. Garg, N. Suenderhauf, and M. Milford, “Don’t look back: Robustifying place categorization for viewpoint- and condition-invariant place recognition,” in 2018 IEEE International Conference on Robotics and Automation (ICRA), Brisbane, QLD, Australia, May 2018, pp. 3645–3652.
  10. ——, “LoST? appearance-invariant place recognition for opposite viewpoints using visual semantics,” in Proceedings of Robotics: Science and Systems, Pittsburgh, PA, USA, June 2018, pp. 1–10.
  11. S. Garg, M. Babu V, T. Dharmasiri, S. Hausler, N. Suenderhauf, S. Kumar, T. Drummond, and M. Milford, “Look no deeper: Recognizing places from opposing viewpoints under varying scene appearance using single-view depth estimation,” in 2019 International Conference on Robotics and Automation (ICRA), Montreal, QC, Canada, May 2019, pp. 4916–4923.
  12. S. Garg, N. Suenderhauf, and M. Milford, “Semantic–geometric visual place recognition: a new perspective for reconciling opposing views,” The International Journal of Robotics Research, vol. 41, no. 6, pp. 573–598, May 2022.
  13. R. Arroyo, P. F. Alcantarilla, L. M. Bergasa, J. J. Yebes, and S. Gámez, “Bidirectional loop closure detection on panoramas for visual navigation,” in 2014 IEEE Intelligent Vehicles Symposium Proceedings, Dearborn, MI, USA, June 2014, pp. 1378–1383.
  14. G. Kim, S. Choi, and A. Kim, “Scan context++: Structural place recognition robust to rotation and lateral variations in urban environments,” IEEE Transactions on Robotics, vol. 38, no. 3, pp. 1856–1874, June 2022.
  15. A. Chapoulie, P. Rives, and D. Filliat, “A spherical representation for efficient visual loop closing,” in 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops), Barcelona, Spain, Nov. 2011, pp. 335–342.
  16. X. Xu, Y. Jiao, S. Lu, X. Ding, R. Xiong, and Y. Wang, “Leveraging BEV representation for 360-degree visual place recognition,” arXiv preprint arXiv:2305.13814, pp. 1–11, May 2023.
  17. T. Cieslewski, E. Stumm, A. Gawel, M. Bosse, S. Lynen, and R. Siegwart, “Point cloud descriptors for place recognition using sparse visual information,” in 2016 IEEE International Conference on Robotics and Automation (ICRA), Stockholm, Sweden, May 2016, pp. 4830–4836.
  18. Y. Ye, T. Cieslewski, A. Loquercio, and D. Scaramuzza, “Place recognition in semi-dense maps: Geometric and learning-based approaches,” in British Machine Vision Conference, London, UK, Sept. 2017, pp. 1–13.
  19. J. Mo and J. Sattar, “A fast and robust place recognition approach for stereo visual odometry using LiDAR descriptors,” in 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Las Vegas, NV, USA, Oct. 2020, pp. 5893–5900.
  20. A. Oertel, T. Cieslewski, and D. Scaramuzza, “Augmenting visual place recognition with structural cues,” IEEE Robotics and Automation Letters, vol. 5, no. 4, pp. 5534–5541, Oct. 2020.
  21. S. Carmichael, A. Buchan, M. Ramanagopal, R. Ravi, R. Vasudevan, and K. A. Skinner, “Dataset and benchmark: Novel sensors for autonomous vehicle perception,” arXiv preprint arXiv:2401.13853, pp. 1–11, Jan. 2024.
  22. H. Jégou, M. Douze, C. Schmid, and P. Pérez, “Aggregating local descriptors into a compact image representation,” in 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, San Francisco, CA, USA, June 2010, pp. 3304–3311.
  23. C. Masone and B. Caputo, “A survey on deep visual place recognition,” IEEE Access, vol. 9, pp. 19 516–19 547, Jan. 2021.
  24. M. Zaffar, A. Khaliq, S. Ehsan, M. Milford, and K. McDonald-Maier, “Levelling the playing field: A comprehensive comparison of visual place recognition approaches under changing conditions,” in IEEE ICRA Workshop on Dataset Generation and Benchmarking of SLAM Algorithms for Robotics and VR/AR, Montreal, Canada, Apr. 2019, pp. 1–8.
  25. M. J. Milford and G. F. Wyeth, “SeqSLAM: Visual route-based navigation for sunny summer days and stormy winter nights,” in 2012 IEEE International Conference on Robotics and Automation, Saint Paul, MN, USA, May 2012, pp. 1643–1649.
  26. E. Pepperell, P. I. Corke, and M. J. Milford, “All-environment visual place recognition with SMART,” in 2014 IEEE International Conference on Robotics and Automation (ICRA), Hong Kong, China, May 2014, pp. 1612–1618.
  27. E. Pepperell, P. Corke, and M. Milford, “Towards persistent visual navigation using SMART,” in Proceedings of the 2013 Australasian Conference on Robotics and Automation, Sydney, Australia, Dec. 2013, pp. 1–9.
  28. S. Garg, T. Fischer, and M. Milford, “Where is your place, visual place recognition?” in Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence (IJCAI), Montreal, Canada, Aug. 2021, pp. 4416–4425.
  29. J. Mo and J. Sattar, “Extending monocular visual odometry to stereo camera systems by scale optimization,” in 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Macau, China, Nov. 2019, pp. 6921–6927.
  30. J. Engel, V. Koltun, and D. Cremers, “Direct sparse odometry,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 40, no. 3, pp. 611–625, Mar. 2018.
  31. G. Kim and A. Kim, “Scan context: Egocentric spatial descriptor for place recognition within 3D point cloud map,” in 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Madrid, Spain, Oct. 2018, pp. 4802–4809.
  32. L. Matthies and S. Shafer, “Error modeling in stereo navigation,” IEEE Journal on Robotics and Automation, vol. 3, no. 3, pp. 239–248, June 1987.
  33. W. Maddern, G. Pascoe, C. Linegar, and P. Newman, “1 year, 1000 km: The Oxford RobotCar dataset,” The International Journal of Robotics Research, vol. 36, no. 1, pp. 3–15, Jan. 2017.
  34. N. Sünderhauf, P. Neubert, and P. Protzel, “Are we there yet? challenging SeqSLAM on a 3000 km journey across all four seasons,” in IEEE ICRA Workshop on Long-Term Autonomy, Karlsruhe, Germany, May 2013, pp. 1–3.

Summary

No one has generated a summary of this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Sign Up to Summarize

Whiteboard

No one has generated a whiteboard explanation for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Sign Up to Generate

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now

Continue Learning

We haven't generated follow-up questions for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now

Collections

Sign up for free to add this paper to one or more collections.

User Circle Single Streamline Icon: https://streamlinehq.com Sign Up