Decision Tree Based Neural Network Design

The structure determination and parameter initialization of an artificial neural network (ANN) is of great importance to its performance. Generally a network is built by trial and error methods with its parameters randomly initialized. Such practice results in low training efficiency and instability of a network. Based on the functional similarity and equivalence of decision tree and ANN, a new network construction and initialization method is proposed, which is called decision tree based neural network (DTBNN for short). The method is mainly based on the entropy net, but has several improvements. Firstly, an initial network structure is determined according to the information of a decision tree. The structure is then optimized by adding neurons and connections in accordance with practical requirements. Secondly, the network is then initialized so that the hyper-plane it represents is much closer to its final version. By these means, the limitations of entropy net are avoided. Network built by this method has shown good performance. Theoretical analysis and experimental results have shown its effectiveness.