For the histogram features, which are usually used in image steganalysis, a category of algorithms based on relative entropy are given to measure the difference between the histograms of the given image and the estimated cover image. And a quantitative steganalysis method based on relative entropy is proposed for JPEG image steganography. Firstly, according to the likelihood ratio test, which is the optimum test for two hypotheses, the superiority of measuring the distance between the two histograms based on relative entropy is analyzed, and two algorithms are given to measure the difference between the two histograms based on relative entropy. Secondly, for the histogram-like features used in the blind steganalysis methods, the new histograms difference features are calculated by the given algorithms, and fed to the support vector regression to train the quantitative steganalyzers for JPEG image steganography. Finally, the proposed steganalysis methods are applied to the quantitative steganalysis of two categories of typical JPEG image steganography: JSteg and the improved F5 steganography. Experimental results show that compared with the other typical algorithms, the steganalyzers based on the histograms difference features calculated by the proposed algorithms have higher accuracy and stability.