Abstract: Regularization method is widely used in solving the inverse problem. An accurate regularization model plays the most important part in solving the inverse problem. The energy constraints should be different for the different types of images and different parts of the same image, but the traditional L1 and L2 models used in the field of image restoration are both based on a single prior assumption. In this paper, according to the defects of the single priori assumption in traditional regularization model, a novel regularization method based on convolution neural network is proposed and applied to image restoration, therefore, the image restoration can be regarded as a classification issue. In this method, the image is partitioned into several blocks, and the convolution neural network is used to extract and classify the features of sub-block images; then the different forms of the priori regularization constraints are adopted considering the different features of the sub-block images, therefore the regularization method is no longer limited to a single priori assumption. Experiments show that the image restoration results by the regularization method based on convolution neural network are superior to those by the traditional regularization model with a single priori assumption. Therefore the regularization method based on convolution neural network can better restore image, maintain the edge texture characteristic of the image nicely, and has lower computational cost.