A New Supervised Manifold Learning Method

A new supervised manifold learning method is proposed in this paper, in order to present a new strategy to efficiently apply manifold learning and nonlinear dimensionality reduction methods to supervised learning problems. The new method realizes efficient supervised learning mainly based on integrating the topology preserving property of the manifold learning methods (Isomap and LLE) and some prominent properties of support vector machine such as efficiency on middle and small sized data sets and essential capability of support vectors calculated from support vector machine. The method is realized via the following steps: first to apply Isomap or LLE to get the embeddings of the original data set in the low dimensional space; then to obtain support vectors, which are the most significant and intrinsic data for the final classification result, by using support vector machine on these low dimensional embedding data; subsequently to get support vectors in the original high dimensional space based on the corresponding labels of the obtained low dimensional support vectors; finally to apply support vector machine again on these high dimensional support vectors to gain the final classification discriminant function. The good performance of the new method on a series of synthetic and real world data sets verifies the feasibility and efficiency of the method.