Papers
Topics
Authors
Recent
Search
2000 character limit reached

Semantic Invariant Multi-view Clustering with Fully Incomplete Information

Published 22 May 2023 in cs.CV | (2305.12743v2)

Abstract: Robust multi-view learning with incomplete information has received significant attention due to issues such as incomplete correspondences and incomplete instances that commonly affect real-world multi-view applications. Existing approaches heavily rely on paired samples to realign or impute defective ones, but such preconditions cannot always be satisfied in practice due to the complexity of data collection and transmission. To address this problem, we present a novel framework called SeMantic Invariance LEarning (SMILE) for multi-view clustering with incomplete information that does not require any paired samples. To be specific, we discover the existence of invariant semantic distribution across different views, which enables SMILE to alleviate the cross-view discrepancy to learn consensus semantics without requiring any paired samples. The resulting consensus semantics remain unaffected by cross-view distribution shifts, making them useful for realigning/imputing defective instances and forming clusters. We demonstrate the effectiveness of SMILE through extensive comparison experiments with 13 state-of-the-art baselines on five benchmarks. Our approach improves the clustering accuracy of NoisyMNIST from 19.3\%/23.2\% to 82.7\%/69.0\% when the correspondences/instances are fully incomplete. The code could be accessed from https://pengxi.me.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (56)
  1. Q. Wang, M. Chen, F. Nie, and X. Li, “Detecting coherent groups in crowd scenes by multiview clustering,” IEEE transactions on pattern analysis and machine intelligence, vol. 42, no. 1, pp. 46–58, 2018.
  2. C. Xu, D. Tao, and C. Xu, “Multi-view self-paced learning for clustering,” in Twenty-Fourth International Joint Conference on Artificial Intelligence, 2015.
  3. Z. Kang, W. Zhou, Z. Zhao, J. Shao, M. Han, and Z. Xu, “Large-scale multi-view subspace clustering in linear time,” in Proceedings of the AAAI conference on artificial intelligence, vol. 34, no. 04, 2020, pp. 4412–4419.
  4. C. Lu, S. Yan, and Z. Lin, “Convex sparse spectral clustering: Single-view to multi-view,” IEEE Transactions on Image Processing, vol. 25, no. 6, pp. 2833–2843, 2016.
  5. Z. Tao, H. Liu, S. Li, Z. Ding, and Y. Fu, “From ensemble clustering to multi-view clustering,” in International Joint Conference on Artificial Intelligence, 2017.
  6. Q. Wang, Z. Ding, Z. Tao, Q. Gao, and Y. Fu, “Partial multi-view clustering via consistent gan,” in 2018 IEEE International Conference on Data Mining (ICDM).   IEEE, 2018, pp. 1290–1295.
  7. A. Vinokourov, N. Cristianini, and J. Shawe-Taylor, “Inferring a semantic representation of text via cross-language correlation analysis,” Advances in neural information processing systems, vol. 15, 2002.
  8. Y. Li, M. Yang, and Z. Zhang, “A survey of multi-view representation learning,” IEEE transactions on knowledge and data engineering, vol. 31, no. 10, pp. 1863–1883, 2018.
  9. C. Zhang, Y. Liu, and H. Fu, “Ae2-nets: Autoencoder in autoencoder networks,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2019, pp. 2577–2585.
  10. X. Peng, Z. Huang, J. Lv, H. Zhu, and J. T. Zhou, “Comic: Multi-view clustering without parameter selection,” in International conference on machine learning.   PMLR, 2019, pp. 5092–5101.
  11. W. Wang, R. Arora, K. Livescu, and J. Bilmes, “On deep multi-view representation learning,” in International conference on machine learning.   PMLR, 2015, pp. 1083–1092.
  12. M. Yin, W. Huang, and J. Gao, “Shared generative latent representation learning for multi-view clustering,” in Proceedings of the AAAI conference on artificial intelligence, vol. 34, no. 04, 2020, pp. 6688–6695.
  13. Z. Yang, Q. Xu, W. Zhang, X. Cao, and Q. Huang, “Split multiplicative multi-view subspace clustering,” IEEE Transactions on Image Processing, vol. 28, no. 10, pp. 5147–5160, 2019.
  14. H. Yu, J. Tang, G. Wang, and X. Gao, “A novel multi-view clustering method for unknown mapping relationships between cross-view samples,” in Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining, 2021, pp. 2075–2083.
  15. F. Gong, Y. Nie, and H. Xu, “Gromov-wasserstein multi-modal alignment and clustering,” in Proceedings of the 31st ACM International Conference on Information & Knowledge Management, 2022, pp. 603–613.
  16. Z. Huang, P. Hu, J. T. Zhou, J. Lv, and X. Peng, “Partially view-aligned clustering,” Advances in Neural Information Processing Systems, vol. 33, pp. 2892–2902, 2020.
  17. M. Yang, Y. Li, Z. Huang, Z. Liu, P. Hu, and X. Peng, “Partially view-aligned representation learning with noise-robust contrastive loss,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2021, pp. 1134–1143.
  18. A. Karpathy and L. Fei-Fei, “Deep visual-semantic alignments for generating image descriptions,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2015, pp. 3128–3137.
  19. J. Wei, X. Xu, Y. Yang, Y. Ji, Z. Wang, and H. T. Shen, “Universal weighting metric learning for cross-modal matching,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2020, pp. 13 005–13 014.
  20. M. Hu and S. Chen, “Doubly aligned incomplete multi-view clustering,” in Proceedings of the 27th International Joint Conference on Artificial Intelligence, 2018, pp. 2262–2268.
  21. S.-Y. Li, Y. Jiang, and Z.-H. Zhou, “Partial multi-view clustering,” in Proceedings of the AAAI conference on artificial intelligence, vol. 28, no. 1, 2014.
  22. C. Xu, D. Tao, and C. Xu, “Multi-view learning with incomplete views,” IEEE Transactions on Image Processing, vol. 24, no. 12, pp. 5812–5825, 2015.
  23. W. Shao, L. He, and P. S. Yu, “Multiple incomplete views clustering via weighted nonnegative matrix factorization with regularization,” in Machine Learning and Knowledge Discovery in Databases: European Conference, ECML PKDD 2015, Porto, Portugal, September 7-11, 2015, Proceedings, Part I.   Springer, 2015, pp. 318–334.
  24. X. Liu, M. Li, C. Tang, J. Xia, J. Xiong, L. Liu, M. Kloft, and E. Zhu, “Efficient and effective regularized incomplete multi-view clustering,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 43, no. 8, pp. 2634–2646, 2021.
  25. C. Zhang, Y. Cui, Z. Han, J. T. Zhou, H. Fu, and Q. Hu, “Deep partial multi-view learning,” IEEE transactions on pattern analysis and machine intelligence, vol. 44, no. 5, pp. 2402–2415, 2020.
  26. C. Xu, Z. Guan, W. Zhao, H. Wu, Y. Niu, and B. Ling, “Adversarial incomplete multi-view clustering.” in International Joint Conference on Artificial Intelligence, vol. 7, 2019, pp. 3933–3939.
  27. Y. Lin, Y. Gou, X. Liu, J. Bai, J. Lv, and X. Peng, “Dual contrastive prediction for incomplete multi-view representation learning,” IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022.
  28. Y. Lin, Y. Gou, Z. Liu, B. Li, J. Lv, and X. Peng, “Completer: Incomplete multi-view clustering via contrastive prediction,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2021, pp. 11 174–11 183.
  29. H. Tang and Y. Liu, “Deep safe incomplete multi-view clustering: Theorem and algorithm,” in International Conference on Machine Learning.   PMLR, 2022, pp. 21 090–21 110.
  30. M. Yang, Y. Li, P. Hu, J. Bai, J. Lv, and X. Peng, “Robust multi-view clustering with incomplete information,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 45, no. 1, pp. 1055–1069, 2022.
  31. Z. Zhang, L. Liu, F. Shen, H. T. Shen, and L. Shao, “Binary multi-view clustering,” IEEE transactions on pattern analysis and machine intelligence, vol. 41, no. 7, pp. 1774–1782, 2018.
  32. M. Yin, W. Liu, M. Li, T. Jin, and R. Ji, “Cauchy loss induced block diagonal representation for robust multi-view subspace clustering,” Neurocomputing, vol. 427, pp. 84–95, 2021.
  33. G. Andrew, R. Arora, J. Bilmes, and K. Livescu, “Deep canonical correlation analysis,” in International conference on machine learning.   PMLR, 2013, pp. 1247–1255.
  34. T. Zhou, C. Zhang, X. Peng, H. Bhaskar, and J. Yang, “Dual shared-specific multiview subspace clustering,” IEEE transactions on cybernetics, vol. 50, no. 8, pp. 3517–3530, 2019.
  35. F. R. Bach and M. I. Jordan, “Kernel independent component analysis,” Journal of machine learning research, vol. 3, no. Jul, pp. 1–48, 2002.
  36. M.-S. Chen, L. Huang, C.-D. Wang, and D. Huang, “Multi-view clustering in latent embedding space,” in Proceedings of the AAAI conference on artificial intelligence, vol. 34, no. 04, 2020, pp. 3513–3520.
  37. Y. Jiang, Q. Xu, Z. Yang, X. Cao, and Q. Huang, “Dm2c: Deep mixed-modal clustering,” Advances in Neural Information Processing Systems, vol. 32, 2019.
  38. N. Tishby, F. C. Pereira, and W. Bialek, “The information bottleneck method,” arXiv preprint physics/0004057, 2000.
  39. Z. Wan, C. Zhang, P. Zhu, and Q. Hu, “Multi-view information-bottleneck representation learning,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 35, no. 11, 2021, pp. 10 085–10 092.
  40. M. Federici, A. Dutta, P. Forré, N. Kushman, and Z. Akata, “Learning robust representations via multi-view information bottleneck,” in 8th International Conference on Learning Representations, 2020.
  41. C. Xu, D. Tao, and C. Xu, “Large-margin multi-view information bottleneck,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 36, no. 8, pp. 1559–1572, 2014.
  42. Y. Tian, D. Krishnan, and P. Isola, “Contrastive multiview coding,” in Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XI 16.   Springer, 2020, pp. 776–794.
  43. J. Xu, H. Tang, Y. Ren, L. Peng, X. Zhu, and L. He, “Multi-level feature learning for contrastive multi-view clustering,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022, pp. 16 051–16 060.
  44. K. Hassani and A. H. Khasahmadi, “Contrastive multi-view representation learning on graphs,” in International conference on machine learning.   PMLR, 2020, pp. 4116–4126.
  45. H. Wang, X. Guo, Z.-H. Deng, and Y. Lu, “Rethinking minimal sufficient representation in contrastive learning,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022, pp. 16 041–16 050.
  46. P. Vincent, H. Larochelle, I. Lajoie, Y. Bengio, P.-A. Manzagol, and L. Bottou, “Stacked denoising autoencoders: Learning useful representations in a deep network with a local denoising criterion.” Journal of machine learning research, vol. 11, no. 12, 2010.
  47. A. Creswell, T. White, V. Dumoulin, K. Arulkumaran, B. Sengupta, and A. A. Bharath, “Generative adversarial networks: An overview,” IEEE signal processing magazine, vol. 35, no. 1, pp. 53–65, 2018.
  48. J.-Y. Zhu, T. Park, P. Isola, and A. A. Efros, “Unpaired image-to-image translation using cycle-consistent adversarial networks,” in Proceedings of the IEEE international conference on computer vision, 2017, pp. 2223–2232.
  49. Z. Han, C. Zhang, H. Fu, and J. T. Zhou, “Trusted multi-view classification,” in International Conference on Learning Representations, 2020.
  50. A. Krizhevsky, I. Sutskever, and G. E. Hinton, “Imagenet classification with deep convolutional neural networks,” Communications of the ACM, vol. 60, no. 6, pp. 84–90, 2017.
  51. K. Simonyan and A. Zisserman, “Very deep convolutional networks for large-scale image recognition,” in 3rd International Conference on Learning Representations (ICLR 2015).   Computational and Biological Learning Society, 2015.
  52. C. Wah, S. Branson, P. Welinder, P. Perona, and S. Belongie, “The caltech-ucsd birds-200-2011 dataset,” 2011.
  53. Q. Le and T. Mikolov, “Distributed representations of sentences and documents,” in International conference on machine learning.   PMLR, 2014, pp. 1188–1196.
  54. L. Wolf, T. Hassner, and I. Maoz, “Face recognition in unconstrained videos with matched background similarity,” in CVPR 2011.   IEEE, 2011, pp. 529–534.
  55. H. W. Kuhn, “The hungarian method for the assignment problem,” Naval research logistics quarterly, vol. 2, no. 1-2, pp. 83–97, 1955.
  56. H. Hotelling, “Relations between two sets of variates,” Breakthroughs in statistics: methodology and distribution, pp. 162–190, 1992.
Citations (9)
View on Semantic Scholar

Summary

No one has generated a summary of this paper yet.

Sign Up to Summarize

Paper to Video (Beta)

No one has generated a video about this paper yet.

Sign Up to Generate All Videos Subscribe on YouTube

Whiteboard

No one has generated a whiteboard explanation for this paper yet.

Sign Up to Generate

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Generate Now

Continue Learning

We haven't generated follow-up questions for this paper yet.

Generate Now

Collections

Sign up for free to add this paper to one or more collections.

Sign Up