Abstract: Histopathological image can reveal the reason and severity of diseases, which is important for clinical diagnosis. Automatic analysis of histopathological image may release doctor’s burden for manual annotation which can preserve more time for doctors to focus on special and difficult cases. However, the ambiguous relationship between local regions in a histopathological image and histopathological characteristics makes it difficult to construct a computer-aid model. An automatic annotation method for histopathological images based on multiple-instance multiple-label (MIML) learning is proposed, aiming at directly modeling the medical experience of doctors, which suggests that each annotated term associated with an image corresponds to a local visually recognized region. We propose a self-adaptive region cutting method with constraints, to segment each image into several visually disjoint regions, and then perform a feature extraction for each generated region based on texture and inner structures. The whole image is regarded as a bag and regions as instances, thus an image is expressed as a multiple-instance sample. Then we propose a sparse ensemble multiple-instance multiple-label learning algorithm, S-MIMLGP, based on Bayesian learning, and compare it with current multiple-instance single label and multiple-instance multiple-label algorithms. The evaluation on a clinical dataset from the dermatology of a large local hospital shows that the proposed method can yield medically acceptable annotation accuracy, hence indicates its effectiveness.