ISSN 1000-1239 CN 11-1777/TP

计算机研究与发展 ›› 2021, Vol. 58 ›› Issue (11): 2485-2499.doi: 10.7544/issn1000-1239.2021.20200523

• 人工智能 • 上一篇    下一篇

基于空间变换的随机森林算法

关晓蔷1,王文剑1,2,庞继芳1,孟银凤3   

  1. 1(山西大学计算机与信息技术学院 太原 030006);2(计算智能与中文信息处理教育部重点实验室(山西大学) 太原 030006);3(山西大学数学科学学院 太原 030006) (gxq0079@sxu.edu.cn)
  • 出版日期: 2021-11-01
  • 基金资助: 
    国家自然科学基金项目(61876103,61673249,U1805263,62006148);山西省重点研发计划项目(201903D421050);山西省1331工程项目

Space Transformation Based Random Forest Algorithm

Guan Xiaoqiang1, Wang Wenjian1,2, Pang Jifang1, Meng Yinfeng3   

  1. 1(School of Computer and Information Technology, Shanxi University, Taiyuan 030006);2(Key Laboratory of Computational Intelligence and Chinese Information Processing (Shanxi University), Ministry of Education, Taiyuan 030006);3(School of Mathematical Sciences, Shanxi University, Taiyuan 030006)
  • Online: 2021-11-01
  • Supported by: 
    This work was supported by the National Natural Science Foundation of China (61876103, 61673249, U1805263, 62006148), the Key Research and Development Program of Shanxi Province (201903D421050), and the 1331 Engineering Project of Shanxi Province.

摘要: 随机森林是机器学习领域中一种常用的分类算法,具有适用范围广且不易过拟合等优点.为了提高随机森林处理多分类问题的能力,提出一种基于空间变换的随机森林算法(space transformation based random forest algorithm, ST-RF).首先,给出一种考虑优先类别的线性判别分析方法(priority class based linear discriminant analysis, PCLDA),利用针对优先类别的投影矩阵对样本进行空间变换,以增强优先类别样本与其他类别样本的区分效果.进而,将PCLDA方法引入随机森林构建过程中,在为每棵决策树随机选择一个优先类别保证随机森林多样性的基础上,利用PCLDA方法创建侧重于不同优先类别的决策树,以提高单棵决策树的分类准确性,从而实现集成模型整体分类性能的有效提升.最后,在10个标准数据集上对ST-RF算法与7种典型随机森林算法进行比较分析,验证所提算法的有效性,并将基于PCLDA的空间变换策略应用到对比算法中,对改进前后的算法性能进行比较分析.实验结果表明:ST-RF算法在处理多分类问题方面具有明显优势,所提出的空间变换策略具有较强的普适性,可以显著提升原算法的分类性能.

关键词: 随机森林, 优先类别, 线性判别分析, 空间变换, 决策树

Abstract: Random forest is a commonly used classification algorithm in the field of machine learning, which has the advantages of wide application and not easy overfitting. In order to improve the overall performance of random forest in dealing with multi-classification problems, a space transformation based random forest algorithm (ST-RF) is proposed. Firstly, a priority class based linear discriminant analysis (PCLDA) method is designed. On the basis of obtaining the projection matrix for priority class, the discrimination effect between priority class samples and other classes samples is enhanced by spatial transformation. Then, PCLDA method is introduced into the process of random forest construction. By selecting the priority class randomly for each decision tree, the diversity among decision trees in random forests is guaranteed. By using the PCLDA method to create decision trees with different priority classes, the classification accuracy of individual decision tree is improved. Thus, the overall classification performance of the integrated model is effectively improved. By comparing the ST-RF algorithm with seven typical random forest algorithms in 10 standard datasets, the effectiveness of the proposed algorithm is verified. Moreover, the spatial transformation strategy based on PCLDA is applied to the above comparison algorithms, and the performance of the algorithms before and after adding the spatial transformation strategy are compared and analyzed. The experimental results show that ST-RF algorithm has obvious advantages in dealing with multi-classification problems, and the proposed spatial transformation strategy has strong universality, which can significantly improve the classification performance of the original algorithm.

Key words: random forest, priority class, linear discriminant analysis, space transformation, decision tree

中图分类号: