Face Occlusion Detection and Reconstruction

Face occlusions (such as glasses, respirator, scarf, etc.) can degrade the performance of face recognition and face animation evidently. How to remove occlusions on face image quickly and automatically becomes a important problem in face image processing. A face occlusion detection and reconstruction algorithm based on fuzzy principal component analysis (FPCA) is proposed in this paper. The main framework is based on analysis and synthesis techniques. In analysis step, optimal coefficients are estimate from occluded face by project to face space (eigenfaces) in the sense of least-square minimization (LSM). In synthesis step, reconstructed face is obtained by linear combinations of prototypes. Then, difference image is computed from original and reconstructed face, and it is filtered by weights calculated from distance and gray level gap from current pixel to its surrounding neighbors, and normalized to ［0 1］. This difference was used as the probability of being occluded. New face is synthesized by weighted sum of reconstructed face and original face based on this occluding probability. In succeeding iterations, fuzzy PCA in stead of classical PCA is used for analysis and reconstruction, and accumulated error is used for occlusion detection. Experimental results show that the proposed algorithm could detect face occlusion precisely and reconstruct smooth natural face, outperforming classical iterative PCA method.