Papers
Topics
Authors
Recent
Search
2000 character limit reached

Privacy-Preserving Autoencoder for Collaborative Object Detection

Published 29 Feb 2024 in eess.IV | (2402.18864v2)

Abstract: Privacy is a crucial concern in collaborative machine vision where a part of a Deep Neural network (DNN) model runs on the edge, and the rest is executed on the cloud. In such applications, the machine vision model does not need the exact visual content to perform its task. Taking advantage of this potential, private information could be removed from the data insofar as it does not significantly impair the accuracy of the machine vision system. In this paper, we present an autoencoder-style network integrated within an object detection pipeline, which generates a latent representation of the input image that preserves task-relevant information while removing private information. Our approach employs an adversarial training strategy that not only removes private information from the bottleneck of the autoencoder but also promotes improved compression efficiency for feature channels coded by conventional codecs like VVC-Intra. We assess the proposed system using a realistic evaluation framework for privacy, directly measuring face and license plate recognition accuracy. Experimental results show that our proposed method is able to reduce the bitrate significantly at the same object detection accuracy compared to coding the input images directly, while keeping the face and license plate recognition accuracy on the images recovered from the bottleneck features low, implying strong privacy protection. Our code is available at https://github.com/bardia-az/ppa-code.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (75)
  1. Cisco, “Cisco annual Internet report (2018–2023),” Mar. 2020.
  2. ISO/IEC JTC 1/SC29/WG1, “Final call for proposals for JPEG AI,” Jan. 2022. N100095.
  3. W. Gao, S. Liu, X. Xu, M. Rafie, Y. Zhang, and I. Curcio, “Recent standard development activities on video coding for machines,” arXiv:2105.12653, 2021.
  4. L. Duan, J. Liu, W. Yang, T. Huang, and W. Gao, “Video coding for machines: A paradigm of collaborative compression and intelligent analytics,” IEEE Transactions on Image Processing, vol. 29, pp. 8680–8695, 2020.
  5. K.-A. Shim, “A survey of public-key cryptographic primitives in wireless sensor networks,” IEEE Communications Surveys & Tutorials, vol. 18, no. 1, pp. 577–601, 2016.
  6. X. Liu, X. Zhao, Z. Xia, Q. Feng, P. Yu, and J. Weng, “Secure outsourced sift: Accurate and efficient privacy-preserving image sift feature extraction,” IEEE Transactions on Image Processing, vol. 32, pp. 4635–4648, 2023.
  7. S. Hu, Q. Wang, J. Wang, Z. Qin, and K. Ren, “Securing sift: Privacy-preserving outsourcing computation of feature extractions over encrypted image data,” IEEE Transactions on Image Processing, vol. 25, no. 7, pp. 3411–3425, 2016.
  8. Y. Kang, J. Hauswald, C. Gao, A. Rovinski, T. Mudge, J. Mars, and L. Tang, “Neurosurgeon: Collaborative intelligence between the cloud and mobile edge,” ACM SIGARCH Computer Architecture News, vol. 45, no. 1, pp. 615–629, 2017.
  9. A. E. Eshratifar and M. Pedram, “Energy and performance efficient computation offloading for deep neural networks in a mobile cloud computing environment,” in Proc. ACM GLSVLSI, p. 111–116, 2018.
  10. A. E. Eshratifar, M. S. Abrishami, and M. Pedram, “JointDNN: An efficient training and inference engine for intelligent mobile cloud computing services,” IEEE Trans. Mobile Computing, vol. 20, no. 2, pp. 565–576, 2021.
  11. A. Banitalebi-Dehkordi, N. Vedula, J. Pei, F. Xia, L. Wang, and Y. Zhang, “Auto-split: A general framework of collaborative edge-cloud ai,” in Proc. ACM KDD, p. 2543–2553, 2021.
  12. Z. He, T. Zhang, and R. B. Lee, “Model inversion attacks against collaborative inference,” in Proc. 35th Annual Computer Security Applications Conference, p. 148–162, 2019.
  13. B. Azizian and I. V. Bajić, “Privacy-preserving feature coding for machines,” in Proc. PCS, pp. 205–209, 2022.
  14. M. Rabbani and R. Joshi, “An overview of the JPEG 2000 still image compression standard,” Signal Processing: Image Communication, vol. 17, no. 1, pp. 3–48, 2002.
  15. G. J. Sullivan, J.-R. Ohm, W.-J. Han, and T. Wiegand, “Overview of the high efficiency video coding (HEVC) standard,” IEEE Trans. Circuits Syst. Video Technol., vol. 22, no. 12, pp. 1649–1668, 2012.
  16. B. Bross, Y.-K. Wang, Y. Ye, S. Liu, J. Chen, G. J. Sullivan, and J.-R. Ohm, “Overview of the versatile video coding (VVC) standard and its applications,” IEEE Trans. Circuits Syst. Video Technol., vol. 31, no. 10, pp. 3736–3764, 2021.
  17. W. Lin and C.-C. Jay Kuo, “Perceptual visual quality metrics: A survey,” J. Visual Commun. Image Represent., vol. 22, no. 4, pp. 297–312, 2011.
  18. MPEG-CDVS, “Compact descriptors for visual search,” 2015. ISO/IEC JTC 1 15938-13.
  19. MPEG-CDVA, “Compact descriptors for video analysis,” 2019. ISO/IEC JTC 1 15938-15.
  20. I. V. Bajić, W. Lin, and Y. Tian, “Collaborative intelligence: Challenges and opportunities,” in Proc. IEEE ICASSP, pp. 8493–8497, 2021.
  21. N. Shlezinger and I. V. Bajić, “Collaborative inference for AI-empowered IoT devices,” IEEE Internet of Things Magazine, vol. 5, no. 4, pp. 92–98, 2022.
  22. J. Ballé, V. Laparra, and E. P. Simoncelli, “End-to-end optimized image compression,” arXiv:1611.01704, 2016.
  23. J. Ballé, D. Minnen, S. Singh, S. J. Hwang, and N. Johnston, “Variational image compression with a scale hyperprior,” in Proc. ICLR, 2018.
  24. D. Minnen, J. Ballé, and G. D. Toderici, “Joint autoregressive and hierarchical priors for learned image compression,” in NeurIPS, 2018.
  25. Z. Cheng, H. Sun, M. Takeuchi, and J. Katto, “Learned image compression with discretized gaussian mixture likelihoods and attention modules,” in Proc. IEEE/CVF CVPR, June 2020.
  26. Y.-H. Ho, C.-C. Chan, W.-H. Peng, H.-M. Hang, and M. Domański, “ANFIC: Image compression using augmented normalizing flows,” IEEE Open J. Circuits Syst., vol. 2, pp. 613–626, 2021.
  27. S. Ma, X. Zhang, C. Jia, Z. Zhao, S. Wang, and S. Wang, “Image and video compression with neural networks: A review,” IEEE Trans. Circuits Syst. Video Technol., vol. 30, no. 6, pp. 1683–1698, 2020.
  28. D. He, Z. Yang, W. Peng, R. Ma, H. Qin, and Y. Wang, “ELIC: Efficient learned image compression with unevenly grouped space-channel contextual adaptive coding,” in Proc. IEEE/CVF CVPR, 2022.
  29. W. Jiang, J. Yang, Y. Zhai, P. Ning, F. Gao, and R. Wang, “MLIC: Multi-reference entropy model for learned image compression,” arXiv:2211.07273, 2023.
  30. W. Jiang and R. Wang, “MLIC++: Linear complexity multi-reference entropy modeling for learned image compression,” arXiv:2307.15421, 2023.
  31. W. Duan, Z. Chang, C. Jia, S. Wang, S. Ma, L. Song, and W. Gao, “Learned image compression using cross-component attention mechanism,” IEEE Transactions on Image Processing, vol. 32, pp. 5478–5493, 2023.
  32. H. Fu, F. Liang, J. Lin, B. Li, M. Akbari, J. Liang, G. Zhang, D. Liu, C. Tu, and J. Han, “Learned image compression with gaussian-laplacian-logistic mixture model and concatenated residual modules,” IEEE Transactions on Image Processing, vol. 32, pp. 2063–2076, 2023.
  33. T. Chen, H. Liu, Z. Ma, Q. Shen, X. Cao, and Y. Wang, “End-to-end learnt image compression via non-local attention optimization and improved context modeling,” IEEE Transactions on Image Processing, vol. 30, pp. 3179–3191, 2021.
  34. Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, “Image quality assessment: From error visibility to structural similarity,” IEEE Trans. Image Processing, vol. 13, no. 4, pp. 600–612, 2004.
  35. N. Le, H. Zhang, F. Cricri, R. Ghaznavi-Youvalari, and E. Rahtu, “Image coding for machines: an end-to-end learned approach,” in Proc. IEEE ICASSP, pp. 1590–1594, 2021.
  36. N. Le, H. Zhang, F. Cricri, R. Ghaznavi-Youvalari, H. R. Tavakoli, and E. Rahtu, “Learned image coding for machines: A content-adaptive approach,” in Proc. IEEE ICME, pp. 1–6, 2021.
  37. S. R. Alvar and I. V. Bajić, “Pareto-optimal bit allocation for collaborative intelligence,” IEEE Transactions on Image Processing, vol. 30, pp. 3348–3361, 2021.
  38. Z. Yuan, S. Rawlekar, S. Garg, E. Erkip, and Y. Wang, “Feature compression for rate constrained object detection on the edge,” in Proc. IEEE MIPR, pp. 1–6, 2022.
  39. H. Choi and I. V. Bajić, “Deep feature compression for collaborative object detection,” in Proc. IEEE ICIP, pp. 3743–3747, 2018.
  40. H. Choi and I. V. Bajić, “Near-lossless deep feature compression for collaborative intelligence,” in Proc. IEEE MMSP, pp. 1–6, 2018.
  41. A. E. Eshratifar, A. Esmaili, and M. Pedram, “Bottlenet: A deep learning architecture for intelligent mobile cloud computing services,” in Proc. IEEE/ACM ISLPED, pp. 1–6, 2019.
  42. R. A. Cohen, H. Choi, and I. V. Bajić, “Lightweight compression of neural network feature tensors for collaborative intelligence,” in Proc. IEEE ICME, pp. 1–6, 2020.
  43. H. Choi and I. V. Bajić, “Scalable image coding for humans and machines,” IEEE Trans. Image Process., vol. 31, pp. 2739–2754, 2022.
  44. N. Yan, C. Gao, D. Liu, H. Li, L. Li, and F. Wu, “Sssic: Semantics-to-signal scalable image coding with learned structural representations,” IEEE Transactions on Image Processing, vol. 30, pp. 8939–8954, 2021.
  45. O. G. Guleryuz, P. A. Chou, H. Hoppe, D. Tang, R. Du, P. Davidson, and S. Fanello, “Sandwiched image compression: Wrapping neural networks around a standard codec,” in Proc. IEEE ICIP, pp. 3757–3761, 2021.
  46. A. Said, M. K. Singh, and R. Pourreza, “Differentiable bit-rate estimation for neural-based video codec enhancement,” in Proc. PCS, 2022.
  47. O. G. Guleryuz, P. A. Chou, H. Hoppe, D. Tang, R. Du, P. Davidson, and S. Fanello, “Sandwiched image compression: Increasing the resolution and dynamic range of standard codecs,” in Proc. PCS, 2022.
  48. S. R. Alvar and I. V. Bajić, “Multi-task learning with compressible features for collaborative intelligence,” in Proc. IEEE ICIP, pp. 1705–1709, 2019.
  49. F. Mireshghallah, M. Taram, P. Vepakomma, A. Singh, R. Raskar, and H. Esmaeilzadeh, “Privacy in deep learning: A survey,” arXiv:2004.12254, 2020.
  50. X. Liu, L. Xie, Y. Wang, J. Zou, J. Xiong, Z. Ying, and A. V. Vasilakos, “Privacy and security issues in deep learning: A survey,” IEEE Access, vol. 9, pp. 4566–4593, 2021.
  51. A. Boulemtafes, A. Derhab, and Y. Challal, “A review of privacy-preserving techniques for deep learning,” Neurocomputing, vol. 384, pp. 21–45, 2020.
  52. S. Ge, B. Liu, P. Wang, Y. Li, and D. Zeng, “Learning privacy-preserving student networks via discriminative-generative distillation,” IEEE Transactions on Image Processing, vol. 32, pp. 116–127, 2023.
  53. M. Fredrikson, S. Jha, and T. Ristenpart, “Model inversion attacks that exploit confidence information and basic countermeasures,” in Proc. ACM CCS, p. 1322–1333, 2015.
  54. C. Dwork, “Differential privacy: A survey of results,” in Theory and Applications of Models of Computation (M. Agrawal, D. Du, Z. Duan, and A. Li, eds.), pp. 1–19, Springer, 2008.
  55. J. Ryu, Y. Zheng, Y. Gao, S. Abuadbba, J. Kim, D. Won, S. Nepal, H. Kim, and C. Wang, “Can differential privacy practically protect collaborative deep learning inference for the internet of things?,” arXiv:2104.03813, 2021.
  56. J. Wang, J. Zhang, W. Bao, X. Zhu, B. Cao, and P. S. Yu, “Not just privacy: Improving performance of private deep learning in mobile cloud,” in Proc. ACM KDD, p. 2407–2416, 2018.
  57. I. Jarin and B. Eshete, “Pricure: Privacy-preserving collaborative inference in a multi-party setting,” in Proc. ACM IWSPA, p. 25–35, 2021.
  58. Z. He, T. Zhang, and R. B. Lee, “Attacking and protecting data privacy in edge–cloud collaborative inference systems,” IEEE Internet of Things Journal, vol. 8, no. 12, pp. 9706–9716, 2021.
  59. S. R. Alvar and I. V. Bajić, “Scalable privacy in multi-task image compression,” in Proc. IEEE VCIP, pp. 1–5, 2021.
  60. T. M. Cover and J. A. Thomas, Elements of Information Theory. Wiley, 2nd ed., 2006.
  61. N. Tishby, F. C. Pereira, and W. Bialek, “The information bottleneck method,” in Proc. 37th annual Allerton Conference on Communication, Control, and Computing, 1999.
  62. I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio, “Generative adversarial nets,” in Proc. NeurIPS, 2014.
  63. H. Choi and I. V. Bajić, “Deep feature compression for collaborative object detection,” in 2018 25th IEEE International Conference on Image Processing (ICIP), pp. 3743–3747, 2018.
  64. R. C. Gonzalez and R. E. Woods, Digital Image Processing. 2018.
  65. Q. Meng, S. Zhao, Z. Huang, and F. Zhou, “MagFace: A universal representation for face recognition and quality assessment,” in Proc. IEEE/CVF CVPR, pp. 14225–14234, 2021.
  66. G. B. Huang, M. Ramesh, T. Berg, and E. Learned-Miller, “Labeled faces in the wild: A database for studying face recognition in unconstrained environments,” Tech. Rep. 07-49, University of Massachusetts, Amherst, October 2007.
  67. L. Best-Rowden, H. Han, C. Otto, B. F. Klare, and A. K. Jain, “Unconstrained face recognition: Identifying a person of interest from a media collection,” IEEE Trans. Inform. Forensics and Security, vol. 9, no. 12, pp. 2144–2157, 2014.
  68. Y. Taigman, M. Yang, M. Ranzato, and L. Wolf, “Web-scale training for face identification,” in Proc. IEEE/CVF CVPR, 2015.
  69. S. Zherzdev and A. Gruzdev, “LPRNet: License Plate Recognition via Deep Neural Networks,” arXiv:1806.10447, 2018.
  70. Z. Xu, W. Yang, A. Meng, N. Lu, and H. Huang, “Towards end-to-end license plate detection and recognition: A large dataset and baseline,” in Proc. ECCV, pp. 255–271, 2018.
  71. T.-Y. Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ramanan, P. Dollár, and C. L. Zitnick, “Microsoft COCO: Common objects in context,” in Proc. ECCV, Sept. 2014.
  72. T. He, Z. Zhang, H. Zhang, Z. Zhang, J. Xie, and M. Li, “Bag of tricks for image classification with convolutional neural networks,” in Proc. IEEE/CVF CVPR, June 2019.
  73. A. Wieckowski, J. Brandenburg, T. Hinz, C. Bartnik, V. George, G. Hege, C. Helmrich, A. Henkel, C. Lehmann, C. Stoffers, I. Zupancic, B. Bross, and D. Marpe, “VVenC: an open and optimized VVC encoder implementation,” in Proc. IEEE ICME Workshops, pp. 1–2, 2021.
  74. G. Bjøntegaard, “Calculation of average psnr differences between rd-curves,” in VCEG Meeting (ITU-T SG16 Q.6), 2001. VCEG-M33.
  75. C. Hollmann, S. Liu, W. Gao, and X. Xu, “[VCM] on VCM reporting template.” ISO/IEC JTC 1/SC 29/WG2 M56185, Jan. 2021.
Citations (1)
View on Semantic Scholar

Summary

No one has generated a summary of this paper yet.

Sign Up to Summarize

Paper to Video (Beta)

No one has generated a video about this paper yet.

Sign Up to Generate All Videos Subscribe on YouTube

Whiteboard

No one has generated a whiteboard explanation for this paper yet.

Sign Up to Generate

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Generate Now

Continue Learning

We haven't generated follow-up questions for this paper yet.

Generate Now

Authors (2)

  1. Bardia Azizian 
  2. Ivan V. Bajic 

Collections

Sign up for free to add this paper to one or more collections.

Sign Up

Tweets

Sign up for free to view the 1 tweet with 0 likes about this paper.

Sign Up for Free