Papers
Topics
Authors
Recent
Search
2000 character limit reached

Investigating the Effect of Misalignment on Membership Privacy in the White-box Setting

Published 8 Jun 2023 in cs.CR and cs.LG | (2306.05093v2)

Abstract: Machine learning models have been shown to leak sensitive information about their training datasets. Models are increasingly deployed on devices, raising concerns that white-box access to the model parameters increases the attack surface compared to black-box access which only provides query access. Directly extending the shadow modelling technique from the black-box to the white-box setting has been shown, in general, not to perform better than black-box only attacks. A potential reason is misalignment, a known characteristic of deep neural networks. In the shadow modelling context, misalignment means that, while the shadow models learn similar features in each layer, the features are located in different positions. We here present the first systematic analysis of the causes of misalignment in shadow models and show the use of a different weight initialisation to be the main cause. We then extend several re-alignment techniques, previously developed in the model fusion literature, to the shadow modelling context, where the goal is to re-align the layers of a shadow model to those of the target model. We show re-alignment techniques to significantly reduce the measured misalignment between the target and shadow models. Finally, we perform a comprehensive evaluation of white-box membership inference attacks (MIA). Our analysis reveals that internal layer activation-based MIAs suffer strongly from shadow model misalignment, while gradient-based MIAs are only sometimes significantly affected. We show that re-aligning the shadow models strongly improves the former's performance and can also improve the latter's performance, although less frequently. Taken together, our results highlight that on-device deployment increases the attack surface and that the newly available information can be used to build more powerful attacks.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (60)
  1. Deep learning with differential privacy. In Proceedings of the 2016 ACM SIGSAC conference on computer and communications security. 308–318.
  2. Apple. 2021. CSAM Detection - Technical Summary 2021. https://www.apple.com/child-safety/pdf/CSAM_Detection_Technical_Summary.pdf.
  3. Stephen Ashmore and Michael Gashler. 2015. A method for finding similarity between multi-layer perceptrons by Forward Bipartite Alignment. In 2015 International Joint Conference on Neural Networks (IJCNN). IEEE, 1–7.
  4. Hacking smart machines with smarter ones: How to extract meaningful data from machine learning classifiers. International Journal of Security and Networks 10, 3 (2015), 137–150.
  5. Reconstructing training data with informed adversaries. In 2022 IEEE Symposium on Security and Privacy (SP). IEEE, 1138–1156.
  6. Membership inference attacks from first principles. In 2022 IEEE Symposium on Security and Privacy (SP). IEEE, 1897–1914.
  7. Towards training reproducible deep learning models. In Proceedings of the 44th International Conference on Software Engineering. 2202–2214.
  8. Label-only membership inference attacks. In International conference on machine learning. PMLR, 1964–1974.
  9. Correlation inference attacks against machine learning models. arXiv preprint arXiv:2112.08806 (2021).
  10. Understanding real-world threats to deep learning models in Android apps. In Proceedings of the 2022 ACM SIGSAC Conference on Computer and Communications Security. 785–799.
  11. Antreas Dionysiou and Elias Athanasopoulos. 2023. SoK: Membership Inference is Harder Than Previously Thought. Proceedings on Privacy Enhancing Technologies 3 (2023), 286–306.
  12. Calibrating noise to sensitivity in private data analysis. In Theory of Cryptography: Third Theory of Cryptography Conference, TCC 2006, New York, NY, USA, March 4-7, 2006. Proceedings 3. Springer, 265–284.
  13. The role of permutation invariance in linear mode connectivity of neural networks. arXiv preprint arXiv:2110.06296 (2021).
  14. Hugging Face. 2023. The AI community building the future. https://huggingface.co/.
  15. Property inference attacks on fully connected neural networks using permutation invariant representations. In Proceedings of the 2018 ACM SIGSAC conference on computer and communications security. 619–633.
  16. Xavier Glorot and Yoshua Bengio. 2010. Understanding the difficulty of training deep feedforward neural networks. In Proceedings of the thirteenth international conference on artificial intelligence and statistics. JMLR Workshop and Conference Proceedings, 249–256.
  17. Deep learning. MIT press.
  18. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition. 770–778.
  19. Robert Hecht-Nielsen. 1990. On the algebraic structure of feedforward network weight spaces. In Advanced Neural Computers. Elsevier, 129–135.
  20. TAPAS: A toolbox for adversarial privacy auditing of synthetic data. arXiv preprint arXiv:2211.06550 (2022).
  21. A First Look at On-device Models in iOS Apps. ACM Transactions on Software Engineering and Methodology 33, 1 (2023), 1–30.
  22. Auditing differentially private machine learning: How private is private sgd? Advances in Neural Information Processing Systems 33 (2020), 22205–22216.
  23. How to combine membership-inference attacks on multiple updated models. arXiv preprint arXiv:2205.06369 (2022).
  24. Bargav Jayaraman and David Evans. 2019. Evaluating differentially private machine learning in practice. In USENIX Security Symposium.
  25. Diederik P Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014).
  26. Learning multiple layers of features from tiny images. (2009).
  27. Harold W Kuhn. 1955. The Hungarian method for the assignment problem. Naval research logistics quarterly 2, 1-2 (1955), 83–97.
  28. Disparate Vulnerability to Membership Inference Attacks. Proceedings on Privacy Enhancing Technologies 1 (2022), 460–480.
  29. Ya Le and Xuan Yang. 2015. Tiny imagenet visual recognition challenge. CS 231N 7, 7 (2015), 3.
  30. Klas Leino and Matt Fredrikson. 2020. Stolen Memories: Leveraging Model Memorization for Calibrated {{\{{White-Box}}\}} Membership Inference. In 29th USENIX security symposium (USENIX Security 20). 1605–1622.
  31. Influence-directed explanations for deep convolutional networks. In 2018 IEEE international test conference (ITC). IEEE, 1–8.
  32. Convergent learning: Do different neural networks learn the same representations? arXiv preprint arXiv:1511.07543 (2015).
  33. Auditing Membership Leakages of Multi-Exit Networks. In Proceedings of the 2022 ACM SIGSAC Conference on Computer and Communications Security. 1917–1931.
  34. {{\{{ML-Doctor}}\}}: Holistic Risk Assessment of Inference Attacks Against Machine Learning Models. In 31st USENIX Security Symposium (USENIX Security 22). 4525–4542.
  35. Deep Learning Face Attributes in the Wild. In Proceedings of International Conference on Computer Vision (ICCV).
  36. A pragmatic approach to membership inferences on machine learning models. In 2020 IEEE European Symposium on Security and Privacy (EuroS&P). IEEE, 521–534.
  37. Comprehensive privacy analysis of deep learning: Passive and active white-box inference attacks against centralized and federated learning. In 2019 IEEE symposium on security and privacy (SP). IEEE, 739–753.
  38. Adversary instantiation: Lower bounds for differentially private machine learning. In 2021 IEEE Symposium on security and privacy (SP). IEEE, 866–882.
  39. SoK: Comparing Different Membership Inference Attacks with a Comprehensive Benchmark. arXiv preprint arXiv:2307.06123 (2023).
  40. Pytorch: An imperative style, high-performance deep learning library. Advances in neural information processing systems 32 (2019).
  41. Data Privacy and Trustworthy Machine Learning Research Lab. 2020. Processed Texas100 and Purchase100 datasets. https://github.com/privacytrustlab/datasets.
  42. White-box vs black-box: Bayes optimal strategies for membership inference. In International Conference on Machine Learning. PMLR, 5558–5567.
  43. {{\{{Updates-Leak}}\}}: Data Set Inference and Reconstruction Attacks in Online Learning. In 29th USENIX security symposium (USENIX Security 20). 1291–1308.
  44. Ml-leaks: Model and data independent membership inference attacks and defenses on machine learning models. arXiv preprint arXiv:1806.01246 (2018).
  45. Membership inference attacks against machine learning models. In 2017 IEEE symposium on security and privacy (SP). IEEE, 3–18.
  46. Karen Simonyan and Andrew Zisserman. 2014. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014).
  47. Sidak Pal Singh and Martin Jaggi. 2020. Model fusion via optimal transport. Advances in Neural Information Processing Systems 33 (2020), 22045–22055.
  48. Liwei Song and Prateek Mittal. 2021. Systematic evaluation of privacy risks of machine learning models. In 30th USENIX Security Symposium (USENIX Security 21). 2615–2632.
  49. Dropout: a simple way to prevent neural networks from overfitting. The journal of machine learning research 15, 1 (2014), 1929–1958.
  50. Synthetic data–anonymisation groundhog day. In 31st USENIX Security Symposium (USENIX Security 22). 1451–1468.
  51. Anshuman Suri and David Evans. 2022. Formalizing and Estimating Distribution Inference Risks. Proceedings on Privacy Enhancing Technologies 4 (2022), 528–551.
  52. Optimizing mode connectivity via neuron alignment. Advances in Neural Information Processing Systems 33 (2020), 15300–15311.
  53. Demystifying membership inference attacks in machine learning as a service. IEEE Transactions on Services Computing 14, 6 (2019), 2073–2089.
  54. Federated learning with matched averaging. arXiv preprint arXiv:2002.06440 (2020).
  55. Zack Whittaker. 2021. Apple’s CSAM detection tech is under fire — again. TechCrunch, https://techcrunch.com/2021/08/18/apples-csam-detection-tech-is-under-fire-again/.
  56. Enhanced Membership Inference Attacks against Machine Learning Models. In Proceedings of the 2022 ACM SIGSAC Conference on Computer and Communications Security. 3093–3106.
  57. Privacy risk in machine learning: Analyzing the connection to overfitting. In 2018 IEEE 31st computer security foundations symposium (CSF). IEEE, 268–282.
  58. Opacus: User-friendly differential privacy library in PyTorch. arXiv preprint arXiv:2109.12298 (2021).
  59. Bayesian nonparametric federated learning of neural networks. In International conference on machine learning. PMLR, 7252–7261.
  60. Leakage of Dataset Properties in Multi-Party Machine Learning.. In USENIX Security Symposium. 2687–2704.
Citations (2)
View on Semantic Scholar

Summary

No one has generated a summary of this paper yet.

Sign Up to Summarize

Paper to Video (Beta)

No one has generated a video about this paper yet.

Sign Up to Generate All Videos Subscribe on YouTube

Whiteboard

No one has generated a whiteboard explanation for this paper yet.

Sign Up to Generate

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Generate Now

Continue Learning

We haven't generated follow-up questions for this paper yet.

Generate Now

Collections

Sign up for free to add this paper to one or more collections.

Sign Up