高级检索

    基于半监督学习的未知异常检测方法

    Semi-Supervised Learning-Based Method for Unknown Anomaly Detection

    • 摘要: 异常检测旨在识别偏离预期行为模式的数据. 虽然半监督异常检测方法可以充分利用有限的标签数据作为先验知识来提高检测准确性,但是收集到的标记异常(即已知异常)很难覆盖所有类型的异常并且在现实场景中往往存在着一些新型的异常(即未知异常),这些异常可能与已知异常表现出不同的特性,因此难以被现有的半监督异常检测方法识别. 针对该问题,提出了一种基于半监督学习的未知异常检测(semi-supervised unknown anomaly detection, SSUAD)方法,旨在同时识别已知异常和未知异常. 该方法利用闭集分类器对已知异常和正常分类,利用未知异常检测器检测未知异常. 此外,还考虑了异常场景中异常和正常极端不平衡的情况,设计了有效的数据增强方法来扩充异常样本的数量. 在UNSW-NB15和KDDCUP99数据集以及一个真实数据集SQB上进行了实验,实验结果表明,相较于现有的异常检测方法,SSUAD在异常检测性能指标AUC-ROC(area under receiver operating characteristic curve)和AUC-PR(area under precision-recall curve)上都有明显的提升. 证明了SSUAD的有效性和合理性.

       

      Abstract: Anomaly detection aims to identify data that deviates from expected behavior patterns. Despite the potential of semi-supervised anomaly detection methods in enhancing detection accuracy by utilizing a limited amount of labeled data as prior knowledge, the labeled anomalies (i.e., seen anomalies) acquired are unlikely to cover all types of anomalies. In real-world scenarios, novel types of anomalies (i.e., unseen anomalies) often emerge, which may exhibit distinct characteristics from the known anomalies, thereby rendering them challenging to detect using existing semi-supervised anomaly detection methods. To address this issue, we propose a semi-supervised unknown anomaly detection (SSUAD) method, aimed at simultaneously identifying both known and unseen anomalies. This method utilizes a closed-set classifier for the classification of known anomalies and normal instances, and an unknown anomaly detector for the detection of unseen anomalies. Moreover, considering the extreme imbalance between anomalies and normal instances in the anomaly detection scenario, we design an effective data augmentation strategy to increase the number of anomaly samples. Experiments are conducted on UNSW-NB15 and KDDCUP99 datasets, as well as a real-world dataset SQB. The results reveal that, compared with existing anomaly detection methods, SSUAD exhibits significant improvement in the anomaly detection performance metrics AUC-ROC and AUC-PR, thereby verifying the effectiveness and reasonableness of the proposed method.

       

    /

    返回文章
    返回