The HJPS Training Algorithm for Multilayer Feedforward Neural Networks

Based on the Hooke-Jeeves pattern search method of optimization theory, a fast training algorithm, HJPS, is proposed for multilayer feedforward neural networks in this paper. It consists of two alternating steps: exploratory search and pattern move. In the training process, only the changed part of the error function is considered. The results of simulations, including a function approximation problems and a pattern recognition problem, show that the propounded algorithm is remarkably improved compared with BP and other faster algorithms in terms of converging speed and computing time. The high generalization ability of HJPS algorithm is also demonstrated by the experimental results.