It is a challenging work to design a robust color image watermarking scheme against geometric distortions. Moments and moment invariants have become a powerful tool in robust image watermarking owing to their image description capability and geometric invariance property. However, the existing moment-based watermarking schemes were mainly designed for gray images but not for color images, and detection quality and robustness will be lowered when watermark is directly embedded into the luminance component or three color channels of color images. Furthermore, the imperceptibility of the embedded watermark is not well guaranteed. Based on algebra of quaternions and exponent moments theory, a new color image watermarking algorithm robust to geometric distortions is proposed in this paper. We firstly introduce the quaternion exponent moments to deal with the color images in a holistic manner, and it is shown that the quaternion exponent moments can be obtained from the conventional exponent moments of each channel. Then we provide a theoretical framework to construct a set of combined invariants with respect to geometric distortions (rotation, scaling, and translation). And finally, we present a new color image watermarking algorithm robust to geometric distortions, which is based on a set of quaternion exponent moments invariants. Experimental results show that the proposed color image watermarking is not only invisible and robust against common signals processing such as median filter, noise adding, and JPEG compression, but also robust against the geometric distortions.