THE EM LEARNING ALGORITHM FOR MULTILAYER STOCHASTIC PERCEPTRONS

in this paper, a learning algorithm for stochastic neural networks, the em algorithm, is proposed based on differential manifold frameworks. At first, we analyze the dual flat manifold structure of stochastic neural networks, and then give the em algorithm implementation for 3 layers of stochastic perceptrons. A speeding theory of the em algorithm is shown. The em algorithm can be regarded as an iterative dual correction learning algorithm in the frameworks of imcomplete data estimation and the best model architecture choice.