Abstract: Based on the fact that there exist widely-used memory devices which possess polarizing recording property in realizing artificial neural networks, the concept of a stepwise non-linear weight structure with local information dependent learning mode (by Hebbian rule) is proposed. The ability of mapping functions of a neuron with the weights is analyzed by numerating the rank of the equivalent input sample matrix. The convergence of the neuron with the weight increasing non-linearly in the learning process is discussed. With the concept, the main reason of the decline of learning rate in BP (Back Propagation) neural networks, in which Sigmoid activation functions are applied, is also pointed out.