Abstract:
Machine learning models improve prediction by learning from training data, but they may also retain information about individual samples, thereby exposing the data to membership inference and re-identification risks. Existing privacy assessments commonly report model-level results and may overlook differences across subsets and attributes in tabular data. We propose a fluctuation-oriented framework that examines these differences along two local dimensions while retaining a global assessment. At the data-subset level, HDBSCAN and a structural outlier score are used to divide training samples into core, boundary, and noise subsets; an equally sized random subset is included for comparison. Membership inference attacks then quantify how readily each subset can be distinguished from non-member samples. At the data-attribute level, random-forest prediction, together with repeated attribute permutation, is used to measure the statistical dependence between each evaluated attribute and the sensitive label. Quasi-identifiers serve as the evaluated attributes. The overall privacy vulnerability score (
OPVS) combines global membership inference performance with fluctuations across the two local dimensions. Experiments on three tabular datasets and three machine learning models reveal clear local differences. Across nine dataset-model settings, the mean area under the receiver operating characteristic curve (AUC) for membership inference on noise subsets exceeds the aggregate mean for core, boundary, and random subsets by approximately 0.133. These results indicate that global metrics can obscure training regions that are more readily distinguished from non-member samples and attributes that exhibit stronger statistical dependence on sensitive labels, while the framework supports privacy assessment at multiple levels of granularity.