As face recognition technology has been integrated into human daily life, face spoofing detection as a key step before face recognition has attracted more and more attention. For print attack and video attack, we propose a difference quantization local binary pattern (DQ_LBP) algorithm for refining the feature of traditional local binary pattern (LBP) by quantifying the difference between the value of central pixel and its neighborhood pixels. DQ_LBP can extract the difference information between the local pixels without increasing the original dimension of LBP, and thus be able to describe the local texture features of images more accurately. In addition, we use the spatial pyramid (SP) algorithm to calculate the histogram of DQ_LBP features in different color spaces and cascade them into a unified feature vector, so as to obtain more elaborate local color texture information and spatial structure information from the face sample, thus, the fraud face detection performance of the algorithm in this paper has been further improved. Extensive experiments are conducted on three challenging face anti-spoofing databases (CASIA FASD, Replay-Attack, and Replay-Mobile) and show that our algorithm has better performance compared with the state of the art. Moreover, it has great potential in the application of real-time devices.