Abstract: Image segmentation is an ill-posed problem, and accurate image segmentation can be done only if users supply enough constraint information. And the constraint information is always obtained in a user interactive way, and the foreground and background brush are used to label parts of the image pixels. In recent years, due to great progress in graph cut for solving the Markov random field (MRF) problem, more and more foreign researchers have developed many interactive image segmentation tools based on graph cut. Among all these tools, segmentation from a bounding box method is really attractive for its user-friendliness and perspective application. Based on recent popular grid MRF and SuperPixel MRF model, we introduce the higher order potential to the MRF based image segmentation problem. High order potential can capture not only pixel level image segmentation accuracy, but also local range image pixel correlation information. So the image segmentation algorithm performance is greatly enhanced due to the introduction of high order potential of for Gibbs energy function. Compared with pair-wise term MRF, high order MRF model has higher image description precision. Experimental results on image database show that our method outperforms GrabCut method. Finally, we extend our image segmentation from a box method to the video object segmentation problem and get good results.