Exploiting Hybrid Kernel-Based Fuzzy Complementary Mutual Information for Selecting Features

Yuan Zhong; Chen Hongmei; Wang Zhihong; Li Tianrui

doi:10.7544/issn1000-1239.202111272

Journal of Computer Research and Development > 2023 > 60(5): 1111-1120. > DOI: 10.7544/issn1000-1239.202111272

Yuan Zhong, Chen Hongmei, Wang Zhihong, Li Tianrui. Exploiting Hybrid Kernel-Based Fuzzy Complementary Mutual Information for Selecting Features[J]. Journal of Computer Research and Development, 2023, 60(5): 1111-1120. DOI: 10.7544/issn1000-1239.202111272

Citation:

PDF (1371 KB)

Exploiting Hybrid Kernel-Based Fuzzy Complementary Mutual Information for Selecting Features

School of Computing and Artificial Intelligence, Southwest Jiaotong University, Chengdu 611756
National Engineering Laboratory of Integrated Transportation Big Data Application Technology (Southwest Jiaotong University), Chengdu 611756

Funds: This work was supported by the National Natural Science Foundation of China (61976182, 62076171, 61976245, 61876157), the Sichuan Key Research and Development Project (2020YFG0035), and Sichuan Science and Technology Achievements Transfer and Transformation Demonstration Project (2022ZHCG0005).

More Information

Author Bio:
Yuan Zhong: born in 1991. PhD. His main research interests include granular computing and knowledge discovery

Chen Hongmei: born in 1971. PhD, professor, PhD supervisor. Member of CCF. Her main research interests include granular computing, rough set, and data mining

Wang Zhihong: born in 1993. PhD candidate. Her main research interests include granular computing, rough set, and data mining

Li Tianrui: born in 1969. PhD, professor, PhD supervisor. Member of CCF. His main research interests include granular computing, rough set, and data mining
Received Date: December 23, 2021
Revised Date: August 07, 2022
Available Online: February 26, 2023

Graphical Abstract

Abstract

Abstract

Fuzzy rough set theory is currently receiving a lot of attention in the fields of data mining and machine learning. The theory provides an effective tool to overcome the discretization problem and can be applied directly to numerical or mixed attribute data. In the fuzzy rough set model, fuzzy relations are defined to measure the similarity between objects and numerical attribute values no longer need to be discretized. The theory has been successfully applied to many fields such as attribute reduction, rule extraction, cluster analysis and outlier detection. Information entropy has been introduced into fuzzy rough set theory for the representation of fuzzy and uncertainty information, resulting in different forms of fuzzy uncertainty measures such as fuzzy information entropy, fuzzy complementary entropy, and fuzzy mutual information. However, most of the proposed fuzzy mutual information on decisions is non-monotonic, which may lead to a non-convergent learning algorithm. To this end, the fuzzy complementary mutual information on decisions is defined based on the hybrid kernel fuzzy complementary entropy, which is shown to vary monotonically with features. Then, the feature selection method is explored by using the hybrid kernel-based fuzzy complementary mutual information and a corresponding algorithm is designed. Experimental results show that the proposed algorithm can select fewer features and maintain or improve the classification accuracy in most cases.
- fuzzy rough set theory,
- hybrid kernel,
- complementary entropy,
- uncertainty measure,
- feature selection

FullText(HTML)

References (35)

References

[1]	Liang Jiye, Wang Feng, Dang Chuangyin, et al. A group incremental approach to feature selection applying rough set technique[J]. IEEE Transactions on Knowledge and Data Engineering, 2012, 26(2): 294−308
[2]	An Shuang, Hu Qinghua, Pedrycz W, et al. Data-distribution-aware fuzzy rough set model and its application to robust classification[J]. IEEE Transactions on Cybernetics, 2015, 46(12): 3073−3085
[3]	Dai Jianhua, Chen Jiaolong. Feature selection via normative fuzzy information weight with application in biological data classification[J]. Applied Soft Computing, 2020, 92(7): 106299
[4]	Sun Lin, Wang Lanying, Ding Weiping, et al. Feature selection using fuzzy neighborhood entropy-based uncertainty measures for fuzzy neighborhood multigranulation rough sets[J]. IEEE Transactions on Fuzzy Systems, 2021, 29(1): 19−33 doi: 10.1109/TFUZZ.2020.2989098
[5]	Wang Chongzhong, Huang Yang, Ding Weiping, et al. Attribute reduction with fuzzy rough self-information measures[J]. Information Sciences, 2021, 49(5): 68−86
[6]	姚晟,徐风,赵鹏,等. 基于自适应邻域空间粗糙集模型的直觉模糊熵特征选择[J]. 计算机研究与发展,2018,55(4):802−814 doi: 10.7544/issn1000-1239.2018.20160919 Yao Sheng, Xu Feng, Zhao Peng, et al. Intuitionistic fuzzy entropy Feature selection algorithm based on adaptive neighborhood space rough set model[J]. Journal of Computer Research and Development, 2018, 55(4): 802−814 (in Chinese) doi: 10.7544/issn1000-1239.2018.20160919
[7]	Wang Chongzhong, Huang Yang, Shao Mingwen, et al. Feature selection based on neighborhood self-information[J]. IEEE Transactions on Cybernetics, 2020, 50(9): 4031−4042 doi: 10.1109/TCYB.2019.2923430
[8]	Dash M, Liu Huan. Consistency-based search in feature selection[J]. Artificial Intelligence, 2003, 151(1/2): 155−176
[9]	Hu Qinghua, Xie Zongxia, Yu Daren. Hybrid attribute reduction based on a novel fuzzy-rough model and information granulation[J]. Pattern Recognition, 2007, 40(12): 3509−3521 doi: 10.1016/j.patcog.2007.03.017
[10]	Wang Chongzhong, Huang Yang, Shao Mingwen, et al. Uncertainty measures for general fuzzy relations[J]. Fuzzy Sets and Systems, 2019, 360(4): 82−96
[11]	Dubois D, Prade H. Rough fuzzy sets and fuzzy rough sets[J]. International Journal of General System, 1990, 17(2/3): 191−209
[12]	Mi Jusheng, Zhang Wenxiu. An axiomatic characterization of a fuzzy generalization of rough sets[J]. Information Sciences, 2004, 160(1/4): 235−249
[13]	王金波,吴伟志. 双论域上的直觉模糊粗糙集[J]. 模糊系统与数学,2021,35(6):1−13 Wang Jinbo, Wu Weizhi. Intuitionistic fuzzy rough sets over two universes[J]. Fuzzy Systems and Mathematics, 2021, 35(6): 1−13 (in Chinese)
[14]	Yeung D S, Chen Degang, Tsang E C C, et al. On the generalization of fuzzy rough sets[J]. IEEE Transactions on Fuzzy Systems, 2005, 13(3): 343−361 doi: 10.1109/TFUZZ.2004.841734
[15]	Jensen R, Shen Qiang. Fuzzy-rough attribute reduction with application to Web categorization[J]. Fuzzy Sets and Systems, 2004, 141(3): 469−485 doi: 10.1016/S0165-0114(03)00021-6
[16]	Hu Qinghua, Yu Daren, Xie Zongxia. Information-preserving hybrid data reduction based on fuzzy-rough techniques[J]. Pattern Recognition Letters, 2006, 27(5): 414−423 doi: 10.1016/j.patrec.2005.09.004
[17]	Chen Degang, Hu Qinghua, Yang Yongping. Parameterized attribute reduction with Gaussian kernel based fuzzy rough sets[J]. Information Sciences, 2011, 181(23): 5169−5179 doi: 10.1016/j.ins.2011.07.025
[18]	Wang Chongzhong, Qi Yali, Shao Mingwen, et al. A fitting model for feature selection with fuzzy rough sets[J]. IEEE Transactions on Fuzzy Systems, 2017, 25(4): 741−753 doi: 10.1109/TFUZZ.2016.2574918
[19]	Yager R R. Entropy measures under similarity relations[J]. International Journal of General System, 1992, 20(4): 341−358 doi: 10.1080/03081079208945039
[20]	Hu Qinghua, Yu Daren, Xie Zongxia, et al. Fuzzy probabilistic approximation spaces and their information measures[J]. IEEE Transactions on Fuzzy Systems, 2006, 14(2): 191−201 doi: 10.1109/TFUZZ.2005.864086
[21]	Qian Yuhua, Liang Jiye, Wu Weizhi, et al. Information granularity in fuzzy binary GrC model[J]. IEEE Transactions on Fuzzy Systems, 2010, 19(2): 253−264
[22]	Xu Jiucheng, Wang Yun, Xu Keqiang, et al. Feature genes selection using fuzzy rough uncertainty metric for tumor diagnosis[J]. Computational and Mathematical Methods in Medicine, 2019, 27(1): 1−9
[23]	樊雲瑞,张贤勇,杨霁琳. 模糊邻域粗糙集的决策熵不确定性度量[J]. 计算机工程与设计,2021,42(5):1300−1306 doi: 10.16208/j.issn1000-7024.2021.05.015 Fan Yunrui, Zhang Xianyong, Yang Jilin. Uncertainty measurement of decision-entropies based on fuzzy neighborhood rough sets[J]. Computer Engineering and Design, 2021, 42(5): 1300−1306 (in Chinese) doi: 10.16208/j.issn1000-7024.2021.05.015
[24]	Zhang Xiao, Mei Changlin, Chen Degang, et al. Feature selection in mixed data: A method using a novel fuzzy rough set-based information entropy[J]. Pattern Recognition, 2016, 56(8): 1−15
[25]	Lin Yaojin, Hu Qinghua, Liu Jinghua, et al. Streaming feature selection for multilabel learning based on fuzzy mutual information[J]. IEEE Transactions on Fuzzy Systems, 2017, 25(6): 1491−1507 doi: 10.1109/TFUZZ.2017.2735947
[26]	Zhao Junyang, Zhang Zhili, Han Chongzhao, et al. Complement information entropy for uncertainty measure in fuzzy rough set and its applications[J]. Soft Computing, 2015, 19(7): 1997−2010 doi: 10.1007/s00500-014-1387-5
[27]	Wang Chongzhong, Huang Yang, Shao Mingwen, et al. Fuzzy rough set-based attribute reduction using distance measures[J]. Knowledge-Based Systems, 2019, 164(1): 205−212
[28]	Hu Qinghua, Zhang Lei, Chen Degang, et al. Gaussian kernel based fuzzy rough sets: Model, uncertainty measures and applications[J]. International Journal of Approximate Reasoning, 2010, 51(4): 453−471 doi: 10.1016/j.ijar.2010.01.004
[29]	Hu Qinghua, Yu Daren, Pedrycz W, et al. Kernelized fuzzy rough sets and their applications[J]. IEEE Transactions on Knowledge & Data Engineering, 2010, 23(11): 1649−1667
[30]	Hu Qinghua, Zhang Lingjun, Zhou YuCan, et al. Large-scale multimodality attribute reduction with multi-kernel fuzzy rough sets[J]. IEEE Transactions on Fuzzy Systems, 2017, 26(1): 226−238
[31]	Yuan Zhong, Chen Hongmei, Yang Xiaoling, et al. Fuzzy complementary entropy using hybrid-kernel function and its unsupervised attribute reduction[J]. Knowledge-Based Systems, 2021, 231(11): 107398
[32]	Liang Jiye, Chin K S, Dang Chuangyin, et al. A new method for measuring uncertainty and fuzziness in rough set theory[J]. International Journal of General Systems, 2002, 31(4): 331−342 doi: 10.1080/0308107021000013635
[33]	Jensen R, Shen Qiang. New approaches to fuzzy-rough feature selection[J]. IEEE Transactions on Fuzzy Systems, 2009, 17(4): 824−838 doi: 10.1109/TFUZZ.2008.924209
[34]	Chen Degang, Zhao Suyun. Local reduction of decision system with fuzzy rough sets[J]. Fuzzy Sets and Systems, 2010, 161(13): 1871−1883 doi: 10.1016/j.fss.2009.12.010
[35]	Yang Yanyan Song Shiji, Chen Degang, et al. Discernible neighborhood counting based incremental feature selection for heterogeneous data[J]. International Journal of Machine Learning and Cybernetics, 2019, 11(8): 1115−1127

[1]	Guo Yaqing, Wang Wenjian, Su Meihong. An Adaptive Regression Feature Selection Method for Datasets with Outliers[J]. Journal of Computer Research and Development, 2019, 56(8): 1695-1707. DOI: 10.7544/issn1000-1239.2019.20190313
[2]	Xu Hang, Zhang Kai, Wang Wenjian. A Feature Selection Method for Small Samples[J]. Journal of Computer Research and Development, 2018, 55(10): 2321-2330. DOI: 10.7544/issn1000-1239.2018.20170748
[3]	Wang Ling, Meng Jianyao. Dynamic Fuzzy Features Selection Based on Variable Weight[J]. Journal of Computer Research and Development, 2018, 55(5): 893-907. DOI: 10.7544/issn1000-1239.2018.20170503
[4]	Yao Sheng, Xu Feng, Zhao Peng, Ji Xia. Intuitionistic Fuzzy Entropy Feature Selection Algorithm Based on Adaptive Neighborhood Space Rough Set Model[J]. Journal of Computer Research and Development, 2018, 55(4): 802-814. DOI: 10.7544/issn1000-1239.2018.20160919
[5]	Dong Hongbin, Teng Xuyang, Yang Xue. Feature Selection Based on the Measurement of Correlation Information Entropy[J]. Journal of Computer Research and Development, 2016, 53(8): 1684-1695. DOI: 10.7544/issn1000-1239.2016.20160172
[6]	Xu Junling, Zhou Yuming, Chen Lin, Xu Baowen. An Unsupervised Feature Selection Approach Based on Mutual Information[J]. Journal of Computer Research and Development, 2012, 49(2): 372-382.
[7]	Jing Hongfang, Wang Bin, YangYahui, Xu Yan. Category Distribution-Based Feature Selection Framework[J]. Journal of Computer Research and Development, 2009, 46(9): 1586-1593.
[8]	Gao Jun, Wang Shitong, Deng Zhaohong. GPSFM: Generalized Potential Support Features Selection Method[J]. Journal of Computer Research and Development, 2009, 46(1): 41-51.
[9]	Xu Yan, Li Jintao, Wang Bin, Sun Chunming, Zhang Sen. A Study on Constraints for Feature Selection in Text Categorization[J]. Journal of Computer Research and Development, 2008, 45(4): 596-602.
[10]	Liu Tao, Wu Gongyi, Chen Zheng. An Effective Unsupervised Feature Selection Method for Text Clustering[J]. Journal of Computer Research and Development, 2005, 42(3).