ISSN 1000-1239 CN 11-1777/TP

Journal of Computer Research and Development ›› 2015, Vol. 52 ›› Issue (7): 1477-1486.doi: 10.7544/issn1000-1239.2015.20140373

Previous Articles     Next Articles

Second Order Newton’s Method for Training Radial Basis Function Neural Networks

Cai Xun1, Chen Zhi1, Kanishka Tyagi2, Yu Kuan3, Li Ziqiang1, Zhu Bo4   

  1. 1(School of Computer Science and Technology, Shandong University, Jinan 250101);2(Department of Electrical Engineering, University of Texas at Arlington, Arlington, USA 76019);3(School of Material Science and Engineering, Shandong Jianzhu University, Jinan 250101);4(School of Materials Science and Engineering, Shandong University, Jinan 250061)
  • Online:2015-07-01

Abstract: A hybrid two-step second-order batch approach is presented for constructing and training radial basis function (RBF) neural networks. Unlike other RBF neural network learning algorithms, the proposed paradigm uses Newton’s method to train each set of network parameters, i.e. spread parameters, mean vector parameters and weighted distance measure(DM) coefficients and output weights parameters. For efficiently calculating the second-order equations of Newton’s method, all the optimal parameters are found out using orthogonal least squares(OLS) with the multiply optimal learning factors(MOLFs) for training mean vector parameters. The simulation results of the proposed hybrid training algorithm on a real dataset are compared with those of the recursive least square based RBF(RLS-RBF) and Levenberg-Marquardt method based RBF(LM-RBF) training algorithms. Also, the analysis of the training performance for optimization of each set of parameters has been presented. The experimental results show that the proposed hybrid optimal weighted DM training algorithm, which is based on the optimization of the mean vectors, weighted DM coefficients and spread parameters, has significant improvement on training convergence speed compared with that of RLS-RBF and has very less computation cost compared with that of LM-RBF. It confirms that Newton’s method solved by OLS is a significantly valuable method for training the RBF neural network.

Key words: radial basis function (RBF) neural network, Hessian matrix, Newton’s method, orthogonal least squares (OLS), weighted distance measure (weighted DM), multiply optimal learning factors (MOLFs)

CLC Number: