ISSN 1000-1239 CN 11-1777/TP

计算机研究与发展 ›› 2015, Vol. 52 ›› Issue (7): 1477-1486.doi: 10.7544/issn1000-1239.2015.20140373

• 人工智能 • 上一篇    下一篇

二阶Newton法训练径向基函数神经网络的算法研究

蔡珣1,陈智1,Kanishka Tyagi2 , 于宽3,李子强1,朱波4   

  1. 1(山东大学计算机科学与技术学院 济南 250101); 2(德克萨斯大学阿灵顿分校电子工程系 美国阿灵顿 76019); 3(山东建筑大学材料科学与工程学院 济南 250101); 4(山东大学材料科学与工程学院 济南 250061) (caixunzh@sdu.edu.cn)
  • 出版日期: 2015-07-01
  • 基金资助: 
    基金项目:国家自然科学基金项目(51473088);山东省科技攻关项目(2012GGE27069)

Second Order Newton’s Method for Training Radial Basis Function Neural Networks

Cai Xun1, Chen Zhi1, Kanishka Tyagi2, Yu Kuan3, Li Ziqiang1, Zhu Bo4   

  1. 1(School of Computer Science and Technology, Shandong University, Jinan 250101);2(Department of Electrical Engineering, University of Texas at Arlington, Arlington, USA 76019);3(School of Material Science and Engineering, Shandong Jianzhu University, Jinan 250101);4(School of Materials Science and Engineering, Shandong University, Jinan 250061)
  • Online: 2015-07-01

摘要: 提出了一种混合加权距离测量(weighted distance measure,weighted DM)参数的构建和训练RBF(radial basis function)神经网络的两步批处理算法.该算法在引进了DM系数参数的基础上,采用Newton法分别对径向基函数的覆盖参数、均值向量参数、加权距离测度系数以及输出权值进行了优化,并在优化过程中利用OLS(orthogonal least squares)法来求解Newton法的方程组. 通过实验数据,不仅分析了Newton法优化的各个参数向量对RBF网络训练的影响,而且比较了混合优化加权DM与RLS-RBF(recursive least square RBF neural network)网络训练算法的收敛性和计算成本. 所得到的结论表明整合了优化参数的加权DM-RBF网络训练算法收敛速度比RLS-RBF网络训练算法更快,而且具有比LM-RBF(Levenberg-Marquardt RBF)训练算法更小的计算成本,从而说明OLS求解的Newton法对优化RBF网络参数具有重要应用价值.

关键词: 径向基函数神经网络, Hessian矩阵, Newton法, 正交最小二乘法, 网络参数优化, 最优学习因子

Abstract: A hybrid two-step second-order batch approach is presented for constructing and training radial basis function (RBF) neural networks. Unlike other RBF neural network learning algorithms, the proposed paradigm uses Newton’s method to train each set of network parameters, i.e. spread parameters, mean vector parameters and weighted distance measure(DM) coefficients and output weights parameters. For efficiently calculating the second-order equations of Newton’s method, all the optimal parameters are found out using orthogonal least squares(OLS) with the multiply optimal learning factors(MOLFs) for training mean vector parameters. The simulation results of the proposed hybrid training algorithm on a real dataset are compared with those of the recursive least square based RBF(RLS-RBF) and Levenberg-Marquardt method based RBF(LM-RBF) training algorithms. Also, the analysis of the training performance for optimization of each set of parameters has been presented. The experimental results show that the proposed hybrid optimal weighted DM training algorithm, which is based on the optimization of the mean vectors, weighted DM coefficients and spread parameters, has significant improvement on training convergence speed compared with that of RLS-RBF and has very less computation cost compared with that of LM-RBF. It confirms that Newton’s method solved by OLS is a significantly valuable method for training the RBF neural network.

Key words: radial basis function (RBF) neural network, Hessian matrix, Newton’s method, orthogonal least squares (OLS), weighted distance measure (weighted DM), multiply optimal learning factors (MOLFs)

中图分类号: