A Soft-Thresholding Coordinate Descent Algorithm for Solving Truncated Hinge Loss

Substantially lessening the number of support vectors (SV) can improve robustness to outliers and deliver more accurate classifiers, and reduce the training and predicting time of SVM. Among various sparse algorithms, the truncated Hinge loss method can reduce the number of support vectors effectively. However, the formulated optimization problem is non-convex which can not be solved by conventional convex optimization method. Therefore, some researchers change the non-convex problem into a series of convex problems with concave-convex procedure (CCCP) method, not only increasing the additional amount of calculation but also falling into a local optimal solution. In order to circumvent these drawbacks, we propose a CCCP-based soft-thresholding(ST) coordinate descent (CD) algorithm (STCD-CCCP). We solve CCCP sub-stage convex problem with coordinate descent algorithm to improve computational efficiency. Moreover, we use soft-thresholding projection technique in dual SVM problem to reduce bigger number of support vectors as a result of the algorithm setting the coefficient of the classifier to zero. Meanwhile, soft-thresholding projection technique can eliminate the adverse effect of the local optimal solution and improve classification accuracy by selecting appropriate regularization parameter. Extensive experiments on simulation database, UCI database and large-scale real database have verified that the proposed algorithm is correct and effective.