A Machine Unlearning Method via Feature Constraint and Adaptive Loss Balance

With the accelerated advancement of digitization, data elements have become the core driving force for the operation of modern society. However, at the same time, data security issues have become increasingly prominent, with frequent occurrences of data breaches and privacy violations, causing serious losses to individuals, organizations, and even countries. Against this backdrop, the security of data elements has become the focus of attention from all sectors of society, and the issue of data privacy protection in deep learning models has also attracted widespread attention. Among them, machine unlearning, as a key technology for protecting user’s privacy, aims to enable models to remove the influence of specific data while maintaining generalization performance for remaining data, providing an effective solution for protecting the security of data elements in deep learning models. Existing machine unlearning methods are mainly divided into two categories: exact unlearning and approximate unlearning. However, exact unlearning methods need to intervene in the original training process of the models, while approximate unlearning methods find it difficult to strike a balance between unlearning performance and model generalization ability. To address these issues, we propose an approximate unlearning framework based on feature constraints and adaptive loss balancing. We adopt a “forgetting-recovering” machine unlearning framework. First, for the “forgetting” process, in order to mimic the feature outputs of retrained models for the forgetting samples, we use a randomly initialized model that has not been trained on the forgetting samples to guide the feature outputs of the unlearning model, constraining forgetting at the feature level to avoid easily obtaining forgetting data information from the model. Then, a small amount of data is used for fine-tuning to “recover” the generalization performance of models on the remaining data. At the same time, we regard the above machine unlearning framework as a multi-task optimization problem and introduce adaptive loss balance to automatically balance the “forgetting” and “recovering” tasks, preventing the model from “over-forgetting” or “over-recovering”, so that the “forgetting” and “recovering” tasks can be trained relatively balanced and steadily. Extensive experiments on 3 image classification datasets show that our method can effectively forget the forgetting data and achieve optimal performance in multiple metrics.