Multi-View Clustering Algorithm via Eliminating the Complementarity Controversy

Zhao Yuhan; Chen Songcan

doi:10.7544/issn1000-1239.202440426

Journal of Computer Research and Development > 2025 > 62(5): 1216-1225. > DOI: 10.7544/issn1000-1239.202440426 CSTR: 32373.14.issn1000-1239.202440426

Zhao Yuhan, Chen Songcan. Multi-View Clustering Algorithm via Eliminating the Complementarity Controversy[J]. Journal of Computer Research and Development, 2025, 62(5): 1216-1225. DOI: 10.7544/issn1000-1239.202440426

Citation:

PDF (1669 KB)

Multi-View Clustering Algorithm via Eliminating the Complementarity Controversy

Zhao Yuhan,
Chen Songcan^,

College of Computer Science and Technology, Nanjing University of Aeronautics and Astronautics, Nanjing 211106
Key Laboratory of Pattern Analysis and Machine Intelligence (Nanjing University of Aeronautics and Astronautics), Ministry of Industry and Information Technology, Nanjing 211106

Funds: This work was supported by the National Science and Technology Major Project (J2019-IV-0018-0086) and the National Natural Science Foundation of China (62076124).

More Information

Author Bio:
Zhao Yuhan: born in 2000. Master. Student member of CCF. Her main research interests include machine learning and pattern recognition

Chen Songcan: born in 1962. PhD, professor, PhD supervisor. Senior member of CCF. His main research interests include pattern recognition, machine learning, and neural computing
Received Date: June 02, 2024
Revised Date: September 02, 2024
Accepted Date: October 15, 2024
Available Online: October 23, 2024

Graphical Abstract

Abstract

Abstract

Multi-view clustering aims to use heterogeneous information from different views to discover the underlying data structure and divide the samples into clusters. Consistency and complementarity are two key elements that affect the performance of multi-view clustering. Consistency emphasizes the semantic similarity between different views. Complementarity emphasizes the mutual supplementation of specific information within each view. At present, the study of consistency has been relatively in-depth, but the study of complementarity is controversial, in which some methods believe that consistency and complementarity can assist each other, but constraining them to the same feature space actually causes a conflict between them. Other approaches accordingly argue that complementary information should be discarded, but this would result in a waste of information. Intuitively, complementarity should exist. The contribution of this paper is to find that existing methods do not have enough insight into the essence of complementarity, i.e., consistency and complementarity are not independent but entangled with each other, which results in conflict. Motivated by this finding, we realize the separation of the two kinds of information through disentangling, specifically making them located in different feature subspaces instead of the same feature space, thus developing a multi-view clustering algorithm that takes into account both consistency and complementarity, effectively extracting the complementary information while avoiding the conflict between consistency and complementarity. Comparative experiments on standard datasets demonstrate the effectiveness of the proposed algorithm.
- multi-view clustering,
- consistency and complementarity,
- representation disentanglement,
- contrastive learning,
- pseudo-labeling

FullText(HTML)

References (42)

References

[1]	Sun Shiliang. A survey of multi-view machine learning[J]. Neural Computing and Applications, 2013, 23: 2031−2038 doi: 10.1007/s00521-013-1362-6
[2]	Chen Mansheng, Lin Jiaqi, Li Xianglong, et al. Representation learning in multi-view clustering: A literature review[J]. Data Science and Engineering, 2022, 7(3): 225−241 doi: 10.1007/s41019-022-00190-8
[3]	Liu Jing, Jiang Yu, Li Zechao, et al. Partially shared latent factor learning with multiview data[J]. IEEE Transactions on Neural Networks and Learning Systems, 2015, 26(6): 1233−1246 doi: 10.1109/TNNLS.2014.2335234
[4]	Xu Chang, Tao Dacheng, Xu Chao. A survey on multi-view learning[J]. arXiv preprint, arXiv: 1304.5634, 2013
[5]	Andrew G, Arora R, Bilmes J, et al. Deep canonical correlation analysis[C]//Proc of the 30th Int Conf on Machine Learning. New York: ACM, 2013: 1247−1255
[6]	Peng Xi, Huang Zhenyu, Lv Jianchen, et al. COMIC: Multi-view clustering without parameter selection[C]//Proc of the 36th Int Conf on Machine Learning. New York: ACM, 2019: 5092−5101
[7]	于晓,刘慧,林毓秀,等. 一致性引导的自适应加权多视图聚类[J]. 计算机研究与发展,2022,59(7):1496−1508 Yu Xiao, Liu Hui, Lin Yuxiu, et al. Consensus guided auto-weighted multi-view clustering[J]. Journal of Computer Research and Development, 2022, 59(7): 1496−1508 (in Chinese)
[8]	Wang Weiran, Arora R, Livescu K, et al. On deep multi-view representation learning[C]//Proc of the 32nd Int Conf on Machine Learning. New York: ACM, 2015: 1083−1092
[9]	Yang Hao, Mao Hua, Woo W L, et al. Consistency enhancement-based deep multiview clustering via contrastive learning[J]. arXiv preprint, arXiv: 2401.12648, 2024
[10]	Zhang Changqing, Liu Yeqing, Fu Huazhu. AE2-Nets: Autoencoder in autoencoder networks[C]//Proc of the 37th IEEE Conf on Computer Vision and Pattern Recognition. Piscataway, NJ: IEEE, 2019: 2577−2585
[11]	Lin Yijie, Gou Yuanbiao, Liu Zitao, et al. Completer: Incomplete multi-view clustering via contrastive prediction[C]//Proc of the 39th IEEE Conf on Computer Vision and Pattern Recognition. Piscataway, NJ: IEEE, 2021: 11174−11183
[12]	Geng Chuanxing, Han Aiyang, Chen Songcan. View-labels are indispensable: A multifacet complementarity study of multi-view clustering[J]. arXiv preprint, arXiv: 2205.02507, 2022
[13]	Xu Jie, Tang Huayi, Ren Yazhou, et al. Multi-level feature learning for contrastive multi-view clustering[C]//Proc of the 40th IEEE Conf on Computer Vision and Pattern Recognition. Piscataway, NJ: IEEE, 2022: 16051−16060
[14]	Liu Suyuan, Liao Qing, Wang Siwei, et al. Robust and consistent anchor graph learning for multi-view clustering[J]. IEEE Transactions on Knowledge and Data Engineering, 2024, 36(8): 4207−4219
[15]	Jia Xiaodong, Jing Xiaoyuan, Zhu Xiaoke, et al. Semi-supervised multi-view deep discriminant representation learning[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021, 43(7): 2496−2509 doi: 10.1109/TPAMI.2020.2973634
[16]	Li Jing, Gao Quanxue, Wang Qianqian, et al. Orthogonal non-negative tensor factorization based multi-view clustering[C]//Proc of the 37th Int Conf on Neural Information Processing Systems. Cambridge, MA: MIT Press, 2023: 18186−18202
[17]	Hu Menglei, Chen Songcan. Doubly aligned incomplete multi-view clustering[C]//Proc of the 27th Int Joint Conf on Artificial Intelligence. San Francisco, CA: Morgan Kaufmann, 2018: 2262−2268
[18]	Chen Zhe, Wu Xiaojun, Xu Tianyang, et al. Fast self-guided multi-view subspace clustering[J]. IEEE Transactions on Image Processing, 2023, 32: 6514−6525 doi: 10.1109/TIP.2023.3261746
[19]	Luo Shirui, Zhang Changqing, Zhang Wei, et al. Consistent and specific multi-view subspace clustering[C]//Proc of the 32nd AAAI Conf on Artificial Intelligence. Palo Alto, CA: AAAI, 2018: 3730−3737
[20]	Liu Jiyuan, Liu Xinwang, Yang Yuexiang, et al. Contrastive multi-view kernel learning[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023, 45(8): 9552−9566 doi: 10.1109/TPAMI.2023.3253211
[21]	Liu Xinwang. SimpleMKKM: Simple multiple kernel k-means[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023, 45(4): 5174−5186 doi: 10.1109/TPAMI.2022.3198638
[22]	Chen Jie, Mao Hua, Peng Dezhong, et al. Multiview clustering by consensus spectral rotation fusion[J]. IEEE Transactions on Image Processing, 2023, 32: 5153−5166 doi: 10.1109/TIP.2023.3310339
[23]	Lin Zhiping, Kang Zhao. Graph filter-based multi-view attributed graph clustering[C]//Proc of the 30th Int Joint Conf on Artificial Intelligence. San Francisco, CA: Morgan Kaufmann, 2021: 2723−2729
[24]	Li Haobin, Li Yunfan, Yang Mouxing, et al. Incomplete multi-view clustering via prototype-based imputation[C]//Proc of the 32nd Int Joint Conf on Artificial Intelligence. San Francisco, CA: Morgan Kaufmann, 2023: 3911−3919
[25]	Tang Huayi, Liu Yong. Deep safe multi-view clustering: Reducing the risk of clustering performance degradation caused by view increase[C]//Proc of the 40th IEEE Conf on Computer Vision and Pattern Recognition. Piscataway, NJ: IEEE, 2022: 202−211
[26]	Lin Yijie, Gou Yuanbiao, Liu Xiaotian, et al. Dual contrastive prediction for incomplete multi-view representation learning[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023, 45(4): 4447−4461
[27]	Chen Jie, Mao Hua, Woo W L, et al. Deep multiview clustering by contrasting cluster assignments[C]//Proc of the 19th IEEE Int Conf on Computer Vision. Piscataway, NJ: IEEE, 2023: 16752−16761
[28]	Oord A, Li Yazhe, Vinyals O. Representation learning with contrastive predictive coding[J]. arXiv preprint, arXiv: 1807.03748, 2018
[29]	Chen Ting, Kornblith S, Norouzi M, et al. A simple framework for contrastive learning of visual representations[C]//Proc of the 37th Int Conf on Machine Learning. New York: ACM, 2020: 1597−1607
[30]	He Kaiming, Fan Haoqi, Wu Yuxin, et al. Momentum contrast for unsupervised visual representation learning[C]//Proc of the 38th IEEE Conf on Computer Vision and Pattern Recognition. Piscataway, NJ: IEEE, 2020: 9729−9738
[31]	Tian Yonglong, Krishnan D, Isola P. Contrastive multiview coding[C]//Proc of the 16th European Conf on Computer Vision. Berlin: Springer, 2020: 776−794
[32]	Tsai Y H, Wu Yue, Salakhutdinov R, et al. Self-supervised learning from a multi-view perspective[C/OL]//Proc of the 9th Int Conf on Learning Representations. 2021[2024-06-01]. https://openreview.net/forum?id=-bdp_8Itjwp
[33]	Tang Huayi, Liu Yong. Deep safe incomplete multi-view clustering: Theorem and algorithm[C]//Proc of the 39th Int Conf on Machine Learning. New York: ACM, 2022: 21090−21110
[34]	Ji Xu, Henriques J F, Vedaldi A. Invariant information clustering for unsupervised image classification and segmentation[C]//Proc of the 17th IEEE Int Conf on Computer Vision. Piscataway, NJ: IEEE, 2019: 9865−9874
[35]	Van der Maaten L, Hinton G. Visualizing data using t-SNE[J]. Journal of Machine Learning Research, 2008, 9(86): 2579−2605
[36]	Guo Xifeng, Gao Long, Liu Xinwang, et al. Improved deep embedded clustering with local structure preservation[C]//Proc of the 26th Int Joint Conf on Artificial Intelligence. San Francisco, CA: Morgan Kaufmann, 2017: 1753−1759
[37]	Sohn K, Berthelot D, Li C L, et al. FixMatch: Simplifying semi-supervised learning with consistency and confidence[C]//Proc of the 34th Int Conf on Neural Information Processing Systems. Cambridge, MA: MIT Press, 2020: 596−608
[38]	Li Yeqing, Nie Feiping, Huang Heng, et al. Large-scale multi-view spectral clustering via bipartite graph[C]//Proc of the 29th AAAI Conf on Artificial Intelligence. Palo Alto, CA: AAAI, 2015: 2750−2756
[39]	Yang Yi, Newsam S. Bag-of-visual-words and spatial extensions for land-use classification[C]//Proc of the 18th SIGSPATIAL Int Conf on Advances in Geographic Information systems. New York: ACM, 2010: 270−279
[40]	Li Feifei, Perona P. A Bayesian hierarchical model for learning natural scene categories[C]//Proc of the 2005 IEEE Computer Society Conf on Computer Vision and Pattern Recognition. Piscataway, NJ: IEEE, 2005: 524−531
[41]	Zhang Zheng, Liu Li, Shen Fumin, et al. Binary multi-view clustering[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2019, 41(7): 1774−1782 doi: 10.1109/TPAMI.2018.2847335
[42]	Trosten D J, Lokse S, Jenssen R, et al. Reconsidering representation alignment for multi-view clustering[C]//Proc of the 39th IEEE Conf on Computer Vision and Pattern Recognition. Piscataway, NJ: IEEE, 2021: 1255−1265

[1]	Wu Huanhuan, Xie Ruilin, Qiao Yuanxin, Chen Xiang, Cui Zhanqi. Optimizing Deep Neural Network Based on Interpretability Analysis[J]. Journal of Computer Research and Development, 2024, 61(1): 209-220. DOI: 10.7544/issn1000-1239.202220803
[2]	Li Qian, Lin Chenhao, Yang Yulong, Shen Chao, Fang Liming. Adversarial Attacks and Defenses Against Deep Learning Under the Cloud-Edge-Terminal Scenes[J]. Journal of Computer Research and Development, 2022, 59(10): 2109-2129. DOI: 10.7544/issn1000-1239.20220665
[3]	Shen Zhengchen, Zhang Qianli, Zhang Chaofan, Tang Xiangyu, Wang Jilong. Location Privacy Attack Based on Deep Learning[J]. Journal of Computer Research and Development, 2022, 59(2): 390-402. DOI: 10.7544/issn1000-1239.20200843
[4]	Gu Mianxue, Sun Hongyu, Han Dan, Yang Su, Cao Wanying, Guo Zhen, Cao Chunjie, Wang Wenjie, Zhang Yuqing. Software Security Vulnerability Mining Based on Deep Learning[J]. Journal of Computer Research and Development, 2021, 58(10): 2140-2162. DOI: 10.7544/issn1000-1239.2021.20210620
[5]	Wang Huijiao, Cong Peng, Jiang Hua, Wei Yongzhuang. Security Analysis of SIMON32/64 Based on Deep Learning[J]. Journal of Computer Research and Development, 2021, 58(5): 1056-1064. DOI: 10.7544/issn1000-1239.2021.20200900
[6]	Zhou Chunyi, Chen Dawei, Wang Shang, Fu Anmin, Gao Yansong. Research and Challenge of Distributed Deep Learning Privacy and Security Attack[J]. Journal of Computer Research and Development, 2021, 58(5): 927-943. DOI: 10.7544/issn1000-1239.2021.20200966
[7]	Chen Jinyin, Chen Yipeng, Chen Yiming, Zheng Haibin, Ji Shouling, Shi Jie, Cheng Yao. Fairness Research on Deep Learning[J]. Journal of Computer Research and Development, 2021, 58(2): 264-280. DOI: 10.7544/issn1000-1239.2021.20200758
[8]	Wang Ruiqin, Wu Zongda, Jiang Yunliang, Lou Jungang. An Integrated Recommendation Model Based on Two-stage Deep Learning[J]. Journal of Computer Research and Development, 2019, 56(8): 1661-1669. DOI: 10.7544/issn1000-1239.2019.20190178
[9]	Zhou Yucong, Liu Yi, Wang Rui. Training Deep Neural Networks for Image Applications with Noisy Labels by Complementary Learning[J]. Journal of Computer Research and Development, 2017, 54(12): 2649-2659. DOI: 10.7544/issn1000-1239.2017.20170637
[10]	Zhang Lei, Zhang Yi. Big Data Analysis by Infinite Deep Neural Networks[J]. Journal of Computer Research and Development, 2016, 53(1): 68-79. DOI: 10.7544/issn1000-1239.2016.20150663