Citation: | He Qiuyan, Deng Minghua. Survey of Universal Domain Adaptation[J]. Journal of Computer Research and Development, 2024, 61(1): 120-144. DOI: 10.7544/issn1000-1239.202220935 |
Domain adaptation relaxes the assumption that test data follows the same distribution as training data in traditional machine learning, and transfers knowledge from label-rich source domain to label-scarce target domain when domain bias exists. However, most domain adaptation methods make assumptions about the relative relationship between the label space of source domain and target domain which limits the application of these methods in reality. Therefore, universal domain adaptation is proposed to label target data without prior information of label spaces of source domain and target domain. Generally, a universal domain adaptation model needs to solve several problems. Firstly, it is desired to judge whether a target sample belongs to classes in source domain. Then it ought to transfer knowledge of common classes shared by source domain and target domain to label target samples, as well as dealing with the domain bias and potential category shift. Due to its practicability, universal domain adaptation has gained more and more attention now. Many branches of universal domain adaptation have been proposed, and researchers are trying to apply it onto datasets different from images. In this review, we conduct an exhaustive survey of universal domain adaptation methods till now from the point of problem setting and strategies. Then we compare typical methods and analyze the difficulties in the universal domain adaptation. After that, we introduce current applications of universal domain adaptation. Meanwhile, we analyze relevant research areas briefly. Lastly, we discuss the problems of universal domain adaptation that can be studied further.
[1] |
Pan Jialin Sinno, Yang Qiang. A survey on transfer learning[J]. IEEE Transactions on Knowledge and Data Engineering, 2010, 22(10): 1345−1359 doi: 10.1109/TKDE.2009.191
|
[2] |
范苍宁,刘鹏,肖婷,等. 深度域适应综述:一般情况与复杂情况[J]. 自动化学报,2021,47(3):515−548 doi: 10.16383/j.aas.c200238
Fan Cangning, Liu Peng, Xiao Ting, et al. A review of deep domain adaptation: General situation and complex situation[J]. Acta Automatica Sinica, 2021, 47(3): 515−548 (in Chinese) doi: 10.16383/j.aas.c200238
|
[3] |
Cao Zhangjie, Long Mingsheng, Wang Jianmin, et al. Partial transfer learning with selective adversarial networks [C]// Proc of the 31st IEEE/CVF Conf on Computer Vision and Pattern Recognition (CVPR). Piscataway, NJ: IEEE, 2018: 2724−2732
|
[4] |
Zhang Jing, Ding Zewei, Li Wanqing, et al. Importance weighted adversarial nets for partial domain adaptation [C]// Proc of the 31st IEEE/CVF Conf on Computer Vision and Pattern Recognition (CVPR). Piscataway, NJ: IEEE, 2018: 8156−8164
|
[5] |
Busto P P, Gall J. Open set domain adaptation [C]// Proc of the 15th IEEE Int Conf on Computer Vision (ICCV). Piscataway, NJ: IEEE, 2017: 754−763
|
[6] |
Saito K, Yamamoto S, Ushiku Y, et al. Open set domain adaptation by backpropagation [C]// Proc of the 15th European Conf on Computer Vision (ECCV). Berlin: Springer, 2018: 156−171
|
[7] |
You Kaichao, Long Mingsheng, Cao Zhangjie, et al. Universal domain adaptation [C]// Proc of the 32nd IEEE/CVF Conf on Computer Vision and Pattern Recognition (CVPR). Piscataway, NJ: IEEE, 2019: 2715−2724
|
[8] |
Farahani A, Voghoei S, Rasheed K, et al. A brief review of domain adaptation [C]// Proc of the 16th Int Conf on Data Science (ICDATA) and the 19th Int Conf on Information & Knowledge Engineering (IKE). Berlin: Springer, 2021: 877−894
|
[9] |
Kundu J N, Venkat N, Rahul M V, et al. Universal source-free domain adaptation [C]// Proc of the 33rd IEEE/CVF Conf on Computer Vision and Pattern Recognition (CVPR). Piscataway, NJ: IEEE, 2020: 4543−4552
|
[10] |
Liang Jian, Hu Dapeng, Feng Jiashi, et al. UMAD: Universal model adaptation under domain and category shift [J]. arXiv preprint, arXiv: 2112.08553, 2021
|
[11] |
Deng Bin, Zhang Yabin, Tang Hui, et al. On universal black-box domain adaptation [J]. arXiv preprint, arXiv: 2104.04665, 2021
|
[12] |
Yin Yueming, Yang Zhen, Hu Haifeng, et al. Universal multi-source domain adaptation for image classification[J]. Pattern Recognition, 2022, 121(1/2): 108238
|
[13] |
Yu Qing, Hashimoto A, Ushiku Y. Divergence optimization for noisy universal domain adaptation [C]// Proc of the 34th IEEE/CVF Conf on Computer Vision and Pattern Recognition (CVPR). Piscataway, NJ: IEEE, 2021: 2515−2524
|
[14] |
Ma Xinhong, Gao Junyu, Xu Changsheng. Active universal domain adaptation [C]// Proc of the 18th IEEE/CVF Int Conf on Computer Vision (ICCV). Piscataway, NJ: IEEE, 2021: 8948−8957
|
[15] |
Chidlovskii B, Sadek A, Wolf C. Universal domain adaptation in ordinal regression [J]. arXiv preprint, arXiv: 2106.11576, 2021
|
[16] |
Fu Bo, Cao Zhangjie, Long Mingsheng, et al. Learning to detect open classes for universal domain adaptation [C]// Proc of the 16th European Conf on Computer Vision (ECCV). Berlin: Springer, 2020: 567−583
|
[17] |
Yin Yueming, Yang Zhen, Wu Xiaofu, et al. Pseudo-margin-based universal domain adaptation[J]. Knowledge-Based Systems, 2021, 229(1/2): 107315
|
[18] |
Cai Ziyun, Song Jie, Zhang Tengfei, et al. Dual contrastive universal adaptation network [C/OL]// Proc of the 22nd IEEE Int Conf on Multimedia and Expo (ICME). Piscataway, NJ: IEEE, 2021[2022-09-09].https://ieeexplore.ieee.org/document/9428238
|
[19] |
Zhang Wei, Li Xiang, Ma Hui, et al. Universal domain adaptation in fault diagnostics with hybrid weighted deep adversarial learning[J]. IEEE Transactions on Industrial Informatics, 2021, 17(12): 7957−7967 doi: 10.1109/TII.2021.3064377
|
[20] |
Lifshitz O, Wolf L. Sample selection for universal domain adaptation [C]// Proc of the 35th AAAI Conf on Artificial Intelligence. Palo Alto, CA: AAAI, 2021: 8592−8600
|
[21] |
Gao Yuan, Chen Peipeng, Gao Yue, et al. Hierarchical feature disentangling network for universal domain adaptation[J]. Pattern Recognition, 2022, 127(1): 108616
|
[22] |
Chen Liang, Lou Yihang, He Jianzhong, et al. Geometric anchor correspondence mining with uncertainty modeling for universal domain adaptation [C]// Proc of the 35th IEEE/CVF Conf on Computer Vision and Pattern Recognition (CVPR). Piscataway, NJ: IEEE, 2022: 16134−16143
|
[23] |
Ren Yu, Cong Yang, Dong Jiahua, et al. Uni3DA: Universal 3D domain adaptation for object recognition[J]. IEEE Transactions on Circuits and Systems for Video Technolny, 2023, 33(1): 379−392 doi: 10.1109/TCSVT.2022.3202213
|
[24] |
Saito K, Kim D, Sclaroff S, et al. Universal domain adaptation through self-supervision [C]// Proc of the 33rd Int Conf on Neural Information Processing Systems. New York: Curran Associates, Inc, 2020: 16282−16292
|
[25] |
Cho J. Synthetic source universal domain adaptation through contrastive learning [J]. Sensors, 2021, 21(22): 7539
|
[26] |
Zhu Ronghang, Li Sheng. Self-supervised universal domain adaptation with adaptive memory separation [C]// Proc of the 21st IEEE Int Conf on Data Mining (ICDM). Piscataway, NJ: IEEE, 2021: 1547−1552
|
[27] |
Li Guangrui, Kang Guoliang, Zhu Yi, et al. Domain consensus clustering for universal domain adaptation [C]// Proc of the 34th IEEE/CVF Conf on Computer Vision and Pattern Recognition (CVPR). Piscataway, NJ: IEEE, 2021: 9752−9761
|
[28] |
Chen Liang, Du Qianjin, Lou Yihang, et al. Mutual nearest neighbor contrast and hybrid prototype self-training for universal domain adaptation [C]// Proc of the 36th AAAI Conf on Artificial Intelligence. Palo Alto, CA: AAAI, 2022: 6248−6257
|
[29] |
Saito K, Saenko K. OVANet: One-vs-all network for universal domain adaptation [C]// Proc of the 18th IEEE/CVF Int Conf on Computer Vision (ICCV). Piscataway, NJ: IEEE, 2021: 8980−8989
|
[30] |
Chen Liang, Lou Yihang, He Jianzhong, et al. Evidential neighborhood contrastive learning for universal domain adaptation [C]// Proc of the 36th AAAI Conf on Artificial Intelligence. Palo Alto, CA: AAAI, 2022: 6258−6267
|
[31] |
Ganin Y, Lempitsky V. Unsupervised domain adaptation by backpropagation [C]// Proc of the 32nd Int Conf on Machine Learning. New York: PMLR, 2015: 1180−1189
|
[32] |
Kang Guoliang, Jiang Lu, Yang Yi, et al. Contrastive adaptation network for unsupervised domain adaptation [C]// Proc of the 32nd IEEE/CVF Conf on Computer Vision and Pattern Recognition (CVPR). Piscataway, NJ: IEEE, 2019: 4888−4897
|
[33] |
Niu Zhenxing, Zhou Mo, Wang Le, et al. Ordinal regression with multiple output CNN for age estimation [C]// Proc of the 29th IEEE Conf on Computer Vision and Pattern Recognition (CVPR). Piscataway, NJ: IEEE, 2016: 4920−4928
|
[34] |
Cao Wenzhi, Mirjalili V, Raschka S. Rank consistent ordinal regression for neural networks with application to age estimation[J]. Pattern Recognition Letters, 2020, 140: 325−331 doi: 10.1016/j.patrec.2020.11.008
|
[35] |
Verma V, Lamb A, Beckham C, et al. Manifold mixup: Better representations by interpolating hidden states [C]// Proc of the 36th Int Conf on Machine Learning. New York: PMLR, 2019: 6438−6447
|
[36] |
Venkateswara H, Eusebio J, Chakraborty S, et al. Deep hashing network for unsupervised domain adaptation [C]// Proc of the 30th IEEE Conf on Computer Vision and Pattern Recognition (CVPR). Piscataway, NJ: IEEE, 2017: 5385−5394
|
[37] |
Saito K, Watanabe K, Ushiku Y, et al. Maximum classifier discrepancy for unsupervised domain adaptation [C]// Proc of the 31st IEEE/CVF Conf on Computer Vision and Pattern Recognition (CVPR). Piscataway, NJ: IEEE, 2018: 3723−3732
|
[38] |
Hartigan J A, Hartigan P M. The dip test of unimodality[J]. The Annals of Statistics, 1985, 13(1): 70−84
|
[39] |
Zhang Hongyi, Cissé M, Dauphin Y N, et al. Mixup: Beyond empirical risk minimization [C/OL]// Proc of the 6th Int Conf on Learning Representations. 2018[2022-09-09].https://openreview.net/forum?id=r1Ddp1-Rb
|
[40] |
Sensoy M, Kaplan L, Kandemir M. Evidential deep learning to quantify classification uncertainty [C]// Proc of the 31st Int Conf on Neural Information Processing Systems. New York: Curran Associates, Inc, 2018: 3183−3193
|
[41] |
Peng Xingchao, Usman B, Kaushik N, et al. Visda: The visual domain adaptation challenge [J]. arXiv preprint, arXiv: 1710.06924, 2017
|
[42] |
Saenko K , Kulis B , Fritz M , et al. Adapting visual category models to new domains [C]// Proc of the 11th European Conf on Computer Vision: Part IV (ECCV’10). Berlin: Springer, 2010: 213–226
|
[43] |
Chen Yuhua, Li Wen, Sakaridis C, et al. Domain adaptive faster R-CNN for object detection in the wild [C]// Proc of the 31st IEEE/CVF Conf on Computer Vision and Pattern Recognition (CVPR). Piscataway, NJ: IEEE, 2018: 3339−3348
|
[44] |
Saito K, Ushiku Y, Harada T, et al. Strong-weak distribution alignment for adaptive object detection [C]// Proc of the 32nd IEEE/CVF Conf on Computer Vision and Pattern Recognition (CVPR). Piscataway, NJ: IEEE, 2019: 6949−6958
|
[45] |
Hsu H K, Yao Chunhan, Tsai Y H, et al. Progressive domain adaptation for object detection [C]// Proc of the 20th IEEE Winter Conf on Applications of Computer Vision (WACV). Piscataway, NJ: IEEE, 2020: 738−746
|
[46] |
Chen Cheng, Dou Qi, Chen Hao, et al. Synergistic image and feature adaptation: Towards cross-modality domain adaptation for medical image segmentation [C]// Proc of the 33rd AAAI Conf on Artificial Intelligence. Palo Alto, CA: AAAI, 2019: 865−872
|
[47] |
Liu Yuang, Zhang Wei, Wang Jun. Source-free domain adaptation for semantic segmentation [C]// Proc of the 34th IEEE/CVF Conf on Computer Vision and Pattern Recognition (CVPR). Piscataway, NJ: IEEE, 2021: 1215−1224
|
[48] |
Bai Lubin, Du Shihong, Zhang Xiuyuan, et al. Domain adaptation for remote sensing image semantic segmentation: An integrated approach of contrastive learning and adversarial learning [J/OL]. IEEE Transactions on Geoscience and Remote Sensing, 2022[2022-09-09].https://ieeexplore.ieee.org/document/9857935/
|
[49] |
He Yufan, Carass A, Zuo Lianrui, et al. Autoencoder based self-supervised test-time adaptation for medical image analysis[J]. Medical Image Analysis, 2021, 72(10): 102136
|
[50] |
Liu Xiaofeng, Yoo C, Xing Fangxu, et al. Deep unsupervised domain adaptation: A review of recent advances and perspectives [J]. arXiv preprint, arXiv: 2208.07422, 2022
|
[51] |
Saltori C, Lathuilière S, Sebe N, et al. SF-UDA3D: Source-free unsupervised domain adaptation for lidar-based 3D object detection [C]// Proc of the 8th Int Conf on 3D Vision (3DV). Piscataway, NJ: IEEE, 2020: 771−780
|
[52] |
Peng Shuwen, Xi Xiaohuan, Wang Cheng, et al. Point-based multilevel domain adaptation for point cloud segmentation [J/OL]. IEEE Geoscience and Remote Sensing Letters, 2022[2022-09-09].https://ieeexplore.ieee.org/document/9272684
|
[53] |
Bian Yikai, Xie Jin, Qian Jianjun. Unsupervised domain adaptive point cloud semantic segmentation [C]// Proc of the 7th Asian Conf on Pattern Recognition (ACPR). Berlin: Springer, 2022: 285−298
|
[54] |
Pan Boxiao, Cao Zhangjie, Adeli E, et al. Adversarial cross-domain action recognition with co-attention [C]// Proc of the 34th AAAI Conf on Artificial Intelligence. Palo Alto, CA: AAAI, 2020: 11815−11822
|
[55] |
Choi J, Sharma G, Schulter S, et al. Shuffle and attend: Video domain adaptation [C]// Proc of the 16th European Conf on Computer Vision (ECCV). Berlin: Springer, 2020: 678−695
|
[56] |
Chen Minhung, Li Baopu, Bao Yingze, et al. Action segmentation with joint self-supervised temporal domain adaptation [C]// Proc of the 33rd IEEE/CVF Conf on Computer Vision and Pattern Recognition (CVPR). Piscataway, NJ: IEEE, 2020: 9451−9460
|
[57] |
Deng Weijian, Zheng Liang, Ye Qixiang, et al. Image-image domain adaptation with preserved self-similarity and domain-dissimilarity for person re-identification [C]// Proc of the 31st IEEE/CVF Conf on Computer Vision and Pattern Recognition (CVPR). Piscataway, NJ: IEEE, 2018: 994−1003
|
[58] |
Mekhazni D, Bhuiyan A, Ekladious G, et al. Unsupervised domain adaptation in the dissimilarity space for person re-identification [C]// Proc of the 16th European Conf on Computer Vision (ECCV). Berlin: Springer, 2020: 159−174
|
[59] |
Chen Qingchao, Liu Yang, Albanie S. Mind-the-gap! Unsupervised domain adaptation for text-video retrieval [C]// Proc of the 35th AAAI Conf on Artificial Intelligence. Palo Alto, CA: AAAI, 2021: 1072−1080
|
[60] |
Chen Pengfei, Li Leida, Wu Jinjian, et al. Unsupervised curriculum domain adaptation for no-reference video quality assessment [C]// Proc of the 18th IEEE/CVF Int Conf on Computer Vision (ICCV). Piscataway, NJ: IEEE, 2021: 5158−5167
|
[61] |
Chen Tsenghung, Liao Yuanhong, Chuang Chingyao Y, et al. Show, adapt and tell: Adversarial training of cross-domain image captioner [C]// Proc of the 15th IEEE Int Conf on Computer Vision (ICCV). Piscataway, NJ: IEEE, 2017: 521−530
|
[62] |
Zhao Wentian, Wu Xinxiao, Luo Jiebo. Cross-domain image captioning via cross-modal retrieval and model adaptation[J]. IEEE Transactions on Image Processing, 2021, 30: 1180−1192 doi: 10.1109/TIP.2020.3042086
|
[63] |
Ghosal D, Hazarika D, Roy A, et al. Kingdom: Knowledge-guided domain adaptation for sentiment analysis [C]// Proc of the 58th Annual Meeting of the Association for Computational Linguistics. Stroudsburg, PA: ACL, 2020: 3198−3210
|
[64] |
Ragab M, Eldele E, Chen Zhenghua, et al. Self-supervised autoregressive domain adaptation for time series data [J/OL]. IEEE Transactions on Neural Networks and Learning Systems, 2022[2022-09-09].https://ieeexplore.ieee.org/document/9804766
|
[65] |
Khurana S, Moritz N, Hori T, et al. Unsupervised domain adaptation for speech recognition via uncertainty driven self-training [C]// Proc of the 46th IEEE Int Conf on Acoustics, Speech and Signal Processing (ICASSP). Piscataway, NJ: IEEE, 2021: 6553−6557
|
[66] |
Tonutti M, Ruffaldi E, Cattaneo A, et al. Robust and subject-independent driving manoeuvre anticipation through domain-adversarial recurrent neural networks[J]. Robotics and Autonomous Systems, 2019, 115: 162−173 doi: 10.1016/j.robot.2019.02.007
|
[67] |
Li Yang, Zheng Wenming, Zong Yuan, et al. A bi-hemisphere domain adversarial neural network model for EEG emotion recognition[J]. IEEE Transactions on Affective Computing, 2021, 12(2): 494−504 doi: 10.1109/TAFFC.2018.2885474
|
[68] |
Ouyang Cheng, Kamnitsas K, Biffi C, et al. Data efficient unsupervised domain adaptation for cross-modality image segmentation [C]// Proc of the 22nd Medical Image Computing and Computer Assisted Intervention (MICCAI) . Berlin: Springer, 2019: 669−677
|
[69] |
Liu Xiaofeng, Xing Fangxu, Yang Chao, et al. Adapting off-the-shelf source segmenter for target medical image segmentation [C]// Proc of the 24th Medical Image Computing and Computer Assisted Intervention (MICCAI). Berlin: Springer, 2021: 549−559
|
[70] |
Hu Jian, Li Xiangjie, Hu Gang, et al. Iterative transfer learning with neural network for clustering and cell type classification in single-cell RNAseq analysis[J]. Nature Machine Intelligence, 2020, 2(10): 607−618 doi: 10.1038/s42256-020-00233-7
|
[71] |
Brbić M, Zitnik M, Wang Sheng, et al. MARS: Discovering novel cell types across heterogeneous single-cell experiments[J]. Nature Methods, 2020, 17(12): 1200−1206 doi: 10.1038/s41592-020-00979-3
|
[72] |
Chen Liang, Zhai Yuyao, He Qiuyan, et al. Integrating deep supervised, self-supervised and unsupervised learning for single-cell RNA-seq clustering and annotation [J]. Genes, 2020, 11(7): 792
|
[73] |
Xie Kaikun, Huang Yu, Zeng Feng, et al. scAIDE: Clustering of large-scale single-cell RNA-seq data reveals putative and rare cell types[J]. NAR Genomics and Bioinformatics, 2020, 2(4): lqaa082 doi: 10.1093/nargab/lqaa082
|
[74] |
Yuan Musu, Chen Liang, Deng Minghua. scMRA: A robust deep learning method to annotate scRNA-seq data with multiple reference datasets[J]. Bioinformatics, 2022, 3(1): 738−745
|
[75] |
Xu Chenling, Lopez R, Mehlman E, et al. Probabilistic harmonization and annotation of single-cell transcriptomics data with deep generative models[J]. Molecular Systems Biolny, 2021, 17(1): e9620 doi: 10.15252/msb.20209620
|
[1] | Liu Qixu, Jin Ze, Chen Canhua, Gao Xinbo, Zheng Ningjun, Fang Yiwei, Feng Yun. Survey on Internet of Things Access Control Security[J]. Journal of Computer Research and Development, 2022, 59(10): 2190-2211. DOI: 10.7544/issn1000-1239.20220510 |
[2] | Yang Fan, Li Fei, Shu Jiwu. Survey on Secure Persistent Memory Storage[J]. Journal of Computer Research and Development, 2020, 57(5): 912-927. DOI: 10.7544/issn1000-1239.2020.20190820 |
[3] | Zhou Yanwei, Yang Qiliang, Yang Bo, Wu Zhenqiang. A Tor Anonymous Communication System with Security Enhancements[J]. Journal of Computer Research and Development, 2014, 51(7): 1538-1546. |
[4] | Yang Bo, Feng Dengguo, Qin Yu, Zhang Qianying, Xi Li, Zheng Changwen. Research on Direct Anonymous Attestation Scheme Based on Trusted Mobile Platform[J]. Journal of Computer Research and Development, 2014, 51(7): 1436-1445. |
[5] | Tan Liang, Meng Weiming, Zhou Mingtian. An Improved Direct Anonymous Attestation Scheme[J]. Journal of Computer Research and Development, 2014, 51(2): 334-343. |
[6] | Zhou Fucai, Yue Xiaohan, Bai Hongbo, Xu Jian. An Efficient Property-Based Attestation Scheme with Flexible Checking Mechanisms of Property Certificate Status[J]. Journal of Computer Research and Development, 2013, 50(10): 2070-2081. |
[7] | Wang Yong, Fang Juan, Ren Xingtian, and Lin Li. Formal Verification of TCG Remote Attestation Protocols Based on Process Algebra[J]. Journal of Computer Research and Development, 2013, 50(2): 325-331. |
[8] | Qian Zhenjiang, Liu Wei, and Huang Hao. OSOSM:Operating System Object Semantics Model and Formal Verification[J]. Journal of Computer Research and Development, 2012, 49(12): 2702-2712. |
[9] | Xu Shiwei, Zhang Huanguo. Formal Security Analysis on Trusted Platform Module Based on Applied π Calculus[J]. Journal of Computer Research and Development, 2011, 48(8): 1421-1429. |
[10] | Zhuo Jiliang, Li Xianxian, Li Jianxin, and Huai Jinpeng. A New Taxonomy of Attacks on Security Protocols and Their Security Evaluation[J]. Journal of Computer Research and Development, 2005, 42(7): 1100-1107. |