Abstract:
Label scarcity remains a major obstacle to deploying machine learning in real applications, because many tasks require costly annotation by domain experts. Active learning reduces this cost by selecting valuable unlabeled instances for expert annotation. Existing methods usually design query criteria from a learner-centric view, such as uncertainty, diversity, committee disagreement, and core-set coverage. However, their gains are not stable across datasets, models, and annotation budgets, and an unsuitable strategy may even underperform random annotation. Meanwhile, conventional active learning often restricts experts to the role of label providers, lacking a collaborative medium that explains why samples are selected and what decision logic the model follows. As a result, expert feedback is difficult to elevate from individual labels into reusable knowledge rules, and the model cannot be effectively aligned with expert domain knowledge. To address these issues, this paper proposes KALE, a knowledge-guided active learning method for explainable human-AI collaboration. KALE uses first-order logic rules as the collaborative medium between the model and experts. It first induces compact logic rules from the current model behavior, and then asks experts to judge their correctness according to domain knowledge. Correct rules are used for deductive weak-label expansion, while incorrect rules are treated as symbolic observations of knowledge-model disagreement and used to drive abductive disagreement querying. KALE further combines class-level rules, local rules, and sparse rule selection to keep expert interaction manageable. Experiments show that KALE achieves more robust improvement than existing active learning baselines, especially under small annotation budgets.