Exploiting Hybrid Kernel-Based Fuzzy Complementary Mutual Information for Selecting Features

Fuzzy rough set theory is currently receiving a lot of attention in the fields of data mining and machine learning. The theory provides an effective tool to overcome the discretization problem and can be applied directly to numerical or mixed attribute data. In the fuzzy rough set model, fuzzy relations are defined to measure the similarity between objects and numerical attribute values no longer need to be discretized. The theory has been successfully applied to many fields such as attribute reduction, rule extraction, cluster analysis and outlier detection. Information entropy has been introduced into fuzzy rough set theory for the representation of fuzzy and uncertainty information, resulting in different forms of fuzzy uncertainty measures such as fuzzy information entropy, fuzzy complementary entropy, and fuzzy mutual information. However, most of the proposed fuzzy mutual information on decisions is non-monotonic, which may lead to a non-convergent learning algorithm. To this end, the fuzzy complementary mutual information on decisions is defined based on the hybrid kernel fuzzy complementary entropy, which is shown to vary monotonically with features. Then, the feature selection method is explored by using the hybrid kernel-based fuzzy complementary mutual information and a corresponding algorithm is designed. Experimental results show that the proposed algorithm can select fewer features and maintain or improve the classification accuracy in most cases.