Papers
Topics
Authors
Recent
Search
2000 character limit reached

Manipulating Trajectory Prediction with Backdoors

Published 21 Dec 2023 in cs.LG, cs.CR, and cs.RO | (2312.13863v2)

Abstract: Autonomous vehicles ought to predict the surrounding agents' trajectories to allow safe maneuvers in uncertain and complex traffic situations. As companies increasingly apply trajectory prediction in the real world, security becomes a relevant concern. In this paper, we focus on backdoors - a security threat acknowledged in other fields but so far overlooked for trajectory prediction. To this end, we describe and investigate four triggers that could affect trajectory prediction. We then show that these triggers (for example, a braking vehicle), when correlated with a desired output (for example, a curve) during training, cause the desired output of a state-of-the-art trajectory prediction model. In other words, the model has good benign performance but is vulnerable to backdoors. This is the case even if the trigger maneuver is performed by a non-casual agent behind the target vehicle. As a side-effect, our analysis reveals interesting limitations within trajectory prediction models. Finally, we evaluate a range of defenses against backdoors. While some, like simple offroad checks, do not enable detection for all triggers, clustering is a promising candidate to support manual inspection to find backdoors.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (61)
  1. Adapt: Efficient multi-agent trajectory prediction with adaptation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 8295–8305, 2023.
  2. Spinning language models: Risks of propaganda-as-a-service and countermeasures. In 2022 IEEE Symposium on Security and Privacy (SP), pages 769–786. IEEE, 2022.
  3. Chris M Bishop. Training with noise is equivalent to tikhonov regularization. Neural computation, 7(1):108–116, 1995.
  4. Strong data augmentation sanitizes poisoning and backdoor attacks without an accuracy tradeoff. In IEEE ICASSP 2021, pages 3855–3859. IEEE, 2021.
  5. nuscenes: A multimodal dataset for autonomous driving. arXiv:1903.11027, 2019.
  6. Advdo: Realistic adversarial attacks for trajectory prediction. In ECCV, pages 36–52, 2022.
  7. Regularization can help mitigate poisoning attacks… with the right hyperparameters. Security and Safety in Machine Learning Systems@ICLR, 2021.
  8. Multipath: Multiple probabilistic anchor trajectory hypotheses for behavior prediction. pages 86–99, 2020.
  9. Baddet: Backdoor attacks on object detection. In European Conference on Computer Vision, pages 396–412. Springer, 2022.
  10. Human trajectory prediction via counterfactual analysis. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 9824–9833, 2021.
  11. Forecast-MAE: Self-supervised pre-training for motion forecasting with masked autoencoders. Proceedings of the IEEE/CVF International Conference on Computer Vision, 2023.
  12. Wild patterns reloaded: A survey of machine learning security against training data poisoning. ACM Computing Surveys, 55(13s):1–39, 2023.
  13. Multimodal trajectory predictions for autonomous driving using deep convolutional networks. ICRA, 2019.
  14. Convolutional social pooling for vehicle trajectory prediction. In IEEE Conference on Computer Vision and Pattern Recognition Workshops, CVPRW, pages 1468–1476, 2018.
  15. Trajectory forecasts in unknown environments conditioned on grid-based plans. ArXiv, abs/2001.00735, 2020.
  16. Multimodal trajectory prediction conditioned on lane-graph traversals. In 5th Annual Conference on Robot Learning, 2021.
  17. Vectornet: Encoding hd maps and agent dynamics from vectorized representation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 11525–11533, 2020.
  18. Gohome: Graph-oriented heatmap output for future motion estimation. In 2022 international conference on robotics and automation (ICRA), pages 9107–9114. IEEE, 2022.
  19. Latent variable sequential set transformers for joint multi-agent motion prediction. In International Conference on Learning Representations, 2021.
  20. Densetnt: End-to-end trajectory prediction from dense goal sets. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 15303–15312, 2021.
  21. Badnets: Identifying vulnerabilities in the ml model supply chain. arXiv, 2017.
  22. Ccil: Context-conditioned imitation learning for urban driving. arXiv preprint arXiv:2305.02649, 2023.
  23. Social gan: Socially acceptable trajectories with generative adversarial networks. In CVPR, June 2018.
  24. A survey on trajectory-prediction methods for autonomous driving. IEEE Transactions on Intelligent Vehicles, 7(3):652–674, 2022.
  25. Gameformer: Game-theoretic modeling and learning of transformer-based interactive prediction and planning for autonomous driving. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 3903–3913, October 2023.
  26. Hdgt: Heterogeneous driving graph transformer for multi-agent trajectory prediction via scene encoding. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2023.
  27. Backdoor attacks on time series: A generative approach. In 2023 IEEE Conference on Secure and Trustworthy Machine Learning (SaTML), pages 392–403. IEEE, 2023.
  28. Dewan Masud Karim. Narrower lanes, safer streets. In Proc. Conf. Regina, pages 1–21, 2015.
  29. Adversarial machine learning-industry perspectives. In 2020 IEEE Security and Privacy Workshops (SPW), pages 69–75. IEEE, 2020.
  30. Curie: A method for protecting svm classifier from poisoning attack. arXiv:1606.01584, 2016.
  31. Desire: Distant future prediction in dynamic scenes with interacting agents. In CVPR, 2017.
  32. Hidden backdoor attack against semantic segmentation models. Security and Safety in Machine Learning Systems@ICML, 2021.
  33. Learning lane graph representations for motion forecasting. In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part II 16, pages 541–556. Springer, 2020.
  34. Composite backdoor attack for deep neural network by mixing existing benign features. In Proceedings of the 2020 ACM SIGSAC Conference on Computer and Communications Security, pages 113–131, 2020.
  35. What do you see? evaluation of explainable artificial intelligence (xai) interpretability through neural backdoors. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining, pages 1027–1035, 2021.
  36. Trojaning attack on neural networks. In Network and Distributed System Sec. Symp., NDSS, pages 45–48, 2018.
  37. Piccolo: Exposing complex backdoors in nlp transformer models. In 2022 IEEE Symposium on Security and Privacy (SP), pages 2025–2042. IEEE, 2022.
  38. Multimodal motion prediction with stacked transformers. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 7577–7586, 2021.
  39. Untargeted backdoor attack against object detection. In ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 1–5. IEEE, 2023.
  40. Dangerous cloaking: Natural trigger based backdoor attacks on object detectors in the physical world. arXiv:2201.08619, 2022.
  41. Trajectory prediction for autonomous driving based on multi-head attention with joint agent-map representation. In 2021 IEEE Intelligent Vehicles Symposium (IV), pages 165–170. IEEE, 2021.
  42. Non-local social pooling for vehicle trajectory prediction. In IEEE Intelligent Vehicles Symposium, IV, pages 975–980, June 2019.
  43. Relational recurrent neural networks for vehicle trajectory prediction. In IEEE Intelligent Transportation Systems Conference, ITSC, 2019.
  44. Attacking deep reinforcement learning with decoupled adversarial policy. IEEE Transactions on Dependable and Secure Computing, 2022.
  45. Wayformer: Motion forecasting via simple & efficient attention networks. In 2023 IEEE International Conference on Robotics and Automation (ICRA), pages 2980–2987. IEEE, 2023.
  46. Scene transformer: A unified architecture for predicting future trajectories of multiple agents. In International Conference on Learning Representations, 2021.
  47. Imposition: Implicit backdoor attack through scenario injection. arXiv preprint arXiv:2306.15755, 2023.
  48. Sophie: An attentive gan for predicting paths compliant to social and physical constraints. In CVPR, June 2019.
  49. Traceback of targeted data poisoning attacks in neural networks. In USENIX Sec. Symp. USENIX Association, 2022.
  50. On defending against label flipping attacks on malware detection system. Neural Computing and Applications, pages 1–20, 2020.
  51. Targeted adversarial attacks against neural network trajectory predictors. In Learning for Dynamics and Control Conference, pages 431–444. PMLR, 2023.
  52. Bypassing backdoor detection algorithms in deep learning. In 2020 IEEE European Symposium on Security and Privacy (EuroS&P), pages 175–183. IEEE, 2020.
  53. Multipath++: Efficient information fusion and trajectory aggregation for behavior prediction. In 2022 International Conference on Robotics and Automation (ICRA), pages 7814–7821. IEEE, 2022.
  54. Backdoor attacks in sequential decision-making agents. Ceur Workshops, 2020.
  55. A temporal-pattern backdoor attack to deep reinforcement learning. In GLOBECOM 2022-2022 IEEE Global Communications Conf., pages 2710–2715. IEEE, 2022.
  56. You are catching my attention: Are vision transformers bad learners under backdoor attacks? In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 24605–24615, 2023.
  57. On adversarial robustness of trajectory prediction for autonomous vehicles. In CVPR, pages 15159–15168, 2022.
  58. Defense against poisoning attack via evaluating training samples using multiple spectral clustering aggregation method. Computers, Materials and Continua, 2019.
  59. Robustness of trajectory prediction models under map-based attacks. In Winter Conf. on Applications of Computer Vision, pages 4541–4550, 2023.
  60. Query-centric trajectory prediction. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023.
  61. Hivt: Hierarchical vector transformer for multi-agent motion prediction. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8823–8833, 2022.

Summary

No one has generated a summary of this paper yet.

Sign Up to Summarize

Paper to Video (Beta)

No one has generated a video about this paper yet.

Sign Up to Generate All Videos Subscribe on YouTube

Whiteboard

No one has generated a whiteboard explanation for this paper yet.

Sign Up to Generate

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Generate Now

Continue Learning

We haven't generated follow-up questions for this paper yet.

Generate Now

Collections

Sign up for free to add this paper to one or more collections.

Sign Up