Locality Preserving Twin Support Vector Machines

For classification problems, support vector machine (SVM) achieves state-of-the-art performance in many real applications. A guarantee of its performance superiority is from the maximization of between-class margin. However, SVM solution does not take into consideration the class distribution and may result in a non-robust solution. Recently, multiple surface support vector machine (MSSVM), as an extension of traditional SVM, has been one of the hot research topics in the field of pattern recognition. Unfortunately, many known MSSVM classification algorithms have not considered the underlying local geometric structure and the descriminant information fully. Therefore, a locality preserving twin support vector machine (LPTSVM) is presented in this paper by introducing the basic theories of the locality preserving projections (LPP) into the MSSVM. This method inherits the characteristic of MSSVM for dealing with the XOR problem, fully considers the local geometric structure between samples and shows the local underlying discriminant information. The linear case, the small sample size case and the nonlinear case of the LPTSVM are discussed in this paper. The LPTSVM optimization problem in the small sample size case is solved by using dimensionality reduction through principal component analysis (PCA) and the problem in the nonlinear case is transformed into an equivalent linear LPTSVM problem under empirical kernel mapping (EKM) method. Experimental results on the artificial and real datasets indicate the effectiveness of the LPTSVM method.