Second Order Newton’s Method for Training Radial Basis Function Neural Networks

A hybrid two-step second-order batch approach is presented for constructing and training radial basis function (RBF) neural networks. Unlike other RBF neural network learning algorithms, the proposed paradigm uses Newton’s method to train each set of network parameters, i.e. spread parameters, mean vector parameters and weighted distance measure(DM) coefficients and output weights parameters. For efficiently calculating the second-order equations of Newton’s method, all the optimal parameters are found out using orthogonal least squares(OLS) with the multiply optimal learning factors(MOLFs) for training mean vector parameters. The simulation results of the proposed hybrid training algorithm on a real dataset are compared with those of the recursive least square based RBF(RLS-RBF) and Levenberg-Marquardt method based RBF(LM-RBF) training algorithms. Also, the analysis of the training performance for optimization of each set of parameters has been presented. The experimental results show that the proposed hybrid optimal weighted DM training algorithm, which is based on the optimization of the mean vectors, weighted DM coefficients and spread parameters, has significant improvement on training convergence speed compared with that of RLS-RBF and has very less computation cost compared with that of LM-RBF. It confirms that Newton’s method solved by OLS is a significantly valuable method for training the RBF neural network.