Abstract: Remote sensing imagery is the only data resource that can acquire information about the ocean, atmosphere, and the Earth’s surface. They have been widely applied in many fields, such as agriculture, military, and urban planning. However, clouds and hazes are inevitable factors when collecting images from satellites, resulting in the loss of information and causing a huge waste of data resources in practical applications. Therefore, how to detect and remove clouds from remote sensing images is a challenging and difficult task that draws a lot of experts’ attention. This paper comprehensively reviews current research progress and summarizes the challenges of cloud detection and removal in remote sensing images. Cloud detection methods are divided into two categories based on whether using deep learning technology, and cloud removal methods are divided into three categories based on whether auxiliary images are used. Then, according to the characteristics of different methods, these methods are reviewed and analyzed systematically, including their advantages and disadvantages, respectively. Afterward, four cloud detection, four thin cloud removal and four thick cloud removal methods are evaluated on two remote sensing datasets. Finally, it discusses future challenges and predicts future research directions. This review paper can provide valuable advice to scientists who are involved in remote sensing image processing.