Abstract: Automatic image annotation has been an active research direction due to its great importance in content-based image retrieval(CBIR). However, the results of existing image annotation methods are still far from practical. Therefore, it is of vital importance to design a high-performance approach which could refine the initial annotations. This paper presents a novel algorithm to solve image annotation refinement problem(IAR) by graph partition and image search engine. Our algorithm focuses on pruning the noisy words in candidate annotation set to enhance image annotation performance. The main idea of the proposed algorithm lies in that candidate annotations are served as graph vertices, and the relevance between two candidate annotations is used to construct the edge weight. Then, the image annotation refinement problem can be converted to the weighted graph partition problem. The edge weight is the annotation similarity weighted by two parameters. Parameter 1 is the relationship between candidate annotation and image visual features, and parameter 2 refers to the importance of candidate annotation in host Web page. Next, we compute max cut of the graph using a heuristic algorithm. After the graph is bi-partitioned, one of the two vertex sets is chosen as final annotations. Experimental results on non-Web images and Web images show that our algorithm outperforms the existing image annotation refinement techniques.