Abstract: Collaborative learning technologies such as federated learning and swarming learning can fully use distributed data to deeply mine the knowledge contained in the data while protecting data privacy. It has a broad application prospect, especially in the medical and health field, where privacy practices and ethical constraints are emphasized. Collaborative efforts always require reliable participants. The performance of the global model in collaborative learning largely depends on participant selection. However, the existing studies need to pay more attention to the heterogeneity of medical participants’ data. As a result, the performance of the global model, including stability, is difficult to be guaranteed. This paper proposes to solve this problem from the perspective of reputation. Through iterative collaborative learning, reputation participants are selected as much as possible to obtain a stable and reliable high-performance global model in collaborative learning. We first propose an AI medical promise (AMP) to describe a medical institution’s data quality and help form a good AI ecosystem in the medical field. Secondly, an iterative collaborative learning framework based on backward selection (colback-learning) is established. The backward selection method is used to iteratively calculate a well-performing and stable global model in polynomial time complexity to complete AMP calculation and accumulation in a single collaborative learning task. In calculating AMP, a scoring function that comprehensively considers global performance indicators is formulated to guide the training of the global model in the medical field. Finally, using real-world medical data to simulate various collaborative learning scenarios, we have shown that the colback-learning can select reliable participants to obtain a global model with good performance. The model’s performance stability is 1.3 to 6 times higher than the state-of-the-art methods. The interpretability of the global model maintains a high consistency with centralized learning.