Real AdaBoost Algorithm for Multi-Class and Imbalanced Classification Problems

The current AdaBoost algorithms often do not consider the priori distribution among different classes. To solve this problem, by transforming the expression of training error from sign function to exponential function, a real AdaBoost algorithm for imbalanced classification problem is proposed to minimize the training error rate, and its error estimation is also given. By the same way, the real AdaBoost algorithm for two-class classification problem could be explained and proved successfully by a new mechanism different from the current explanation of the real AdaBoost algorithm. And it could be extended to multi-class classification problem with similar algorithm flow and formulas to the AdaBoost algorithm for two-class classification problem, which is proved to be consistent with the Bayes optimization deduction method. It is proved that by using the proposed real AdaBoost algorithm for multi-class classification problem, the training error rate decreases while the number of training classifiers increases. Theoretical analysis and experimental results on UCI dataset show the effectiveness of the proposed real AdaBoost algorithm for imbalanced classification problem. Imbalanced classification problems are often transformed to balanced classification problems by adjusting the weights of training samples in real AdaBoost algorithm, but when the priori distribution is very imbalanced, the proposed real AdaBoost algorithm for imbalanced classification problem is more effective than the existing real AdaBoost algorithms.