Circuit Boards Anomaly Detection Based on Background-Foreground Compositional Modeling

Fu Bingfei; Chen Tonglin; Xu Feng; Zhu Lin; Li Bin; Xue Xiangyang

doi:10.7544/issn1000-1239.202330565

Journal of Computer Research and Development > 2025 > 62(1): 144-159. > DOI: 10.7544/issn1000-1239.202330565 CSTR: 32373.14.issn1000-1239.202330565

Fu Bingfei, Chen Tonglin, Xu Feng, Zhu Lin, Li Bin, Xue Xiangyang. Circuit Boards Anomaly Detection Based on Background-Foreground Compositional Modeling[J]. Journal of Computer Research and Development, 2025, 62(1): 144-159. DOI: 10.7544/issn1000-1239.202330565

Citation:

PDF (2748 KB)

Circuit Boards Anomaly Detection Based on Background-Foreground Compositional Modeling

1.
School of Computer Science, Fudan University, Shanghai 200438
2.
Artificial Intelligence Lab, Lenovo Research, Beijing 100094

Funds: This work was supported by the Science and Technology Commission of Shanghai Municipality Program (22511105000) and Lenovo Scientists Program.

More Information

Author Bio:
Fu Bingfei: born in 1993. PhD candidate. His main research interests include video, image information processing and retrieval, and computer vision

Chen Tonglin: born in 1991. PhD candidate. His main research interests include machine learning, deep generative models, and object-centric representation learning

Xu Feng: born in 1984. Master, advisory researcher. His main research interests include pattern recognition and intelligent system, and software engineering

Zhu Lin: born in 1980. PhD, Senior R&D Manager in Artificial Intelligence Lab, Lenovo Research Institute. His main research interests include computer vision and machine learning

Li Bin: born in 1982. PhD, associate professor, PhD supervisor. His main research interests include machine learning and visual intelligence, particularly in compositional scene representation, modeling, and inference

Xue Xiangyang: born in 1968. PhD, professor, PhD supervisor. His main research interests include multimedia information processing and machine learning
Received Date: July 02, 2023
Revised Date: May 28, 2024
Available Online: November 12, 2024

Graphical Abstract

Abstract

Abstract

Anomaly detection aims to detect abnormal samples among many normal samples. In the era of big data, how to apply anomaly detection to real-world scenarios has become one of the most critical problems to consider. Currently, the existing models can hardly cope with dynamic interference such as occlusion, lighting, and color difference in real-world scenarios and cannot quickly migrate application scenarios. We propose a deep learning model based on background-foreground modeling for anomaly detection tasks. Our model first reconstructs the input image into a clean background image without abnormal objects through the feature extraction network and preserves the possible dynamic interference of the image through skip-connection. After obtaining the reconstructed background, this model extracts the position information of abnormal objects through the spatial transformation network, uses an autoencoder to extract latent space representations of the appearance, shape, and presence information of abnormal objects, and reconstructs them. Finally, this model combines the reconstructed abnormal objects and the background image to obtain an overall reconstructed image and realizes anomaly detection by setting a threshold for the presence information of abnormal objects. To validate the effectiveness of the method, we collect data from a real circuit board assembly environment and simulate a scenario with limited annotations in actual production, resulting in the creation of a foreign object in circuit board (FO-CB) dataset for analysis. Additionally, we also conduct experimental validation on the foreign object debris in airport (FOD-A) dataset. The experimental results show that our proposed method performs well on the synthetic dataset and detects all anomalous objects in 9 actual scene data, with a miss rate of down to 0%, and can be applied to real-world circuit board assembly scenarios.
- anomaly detection,
- synthetic dataset,
- compositional modeling,
- generative model,
- multi-stage training

FullText(HTML)

References (54)

References

[1]	韩东明,郭方舟,潘嘉铖,等. 面向时序数据异常检测的可视分析综述[J]. 计算机研究与发展,2018,55(9):1843−1852 doi: 10.7544/issn1000-1239.2018.20180126 Han Dongming, Guo Fangzhou, Pan Jiacheng, et al. Visual analysis for anomaly detection in time-series: A survey[J]. Journal of Computer Research and Development, 2018, 55(9): 1843−1852 (in Chinese) doi: 10.7544/issn1000-1239.2018.20180126
[2]	周小晖,王意洁,徐鸿祚,等. 基于融合学习的无监督多维时间序列异常检测[J]. 计算机研究与发展,2023,60(3):496−508 doi: 10.7544/issn1000-1239.202220490 Zhou Xiaohui, Wang Yijie, Xu Hongzuo, et al. Fusion learning based unsupervised anomaly detection for multi-dimensional time series[J]. Journal of Computer Research and Development, 2023, 60(3): 496−508 (in Chinese) doi: 10.7544/issn1000-1239.202220490
[3]	杨雅辉,杜克明. 全网异常流量簇的检测与确定机制[J]. 计算机研究与发展,2009,46(11):1847−1853 Yang Yahui, Du Keming. Identification of anomalous traffic clusters for network-wide anomaly analysis[J]. Journal of Computer Research and Development, 2009, 46(11): 1847−1853 (in Chinese)
[4]	李晓灿,谢鲲,张大方,等. 基于低秩分解的网络异常检测综述[J]. 计算机研究与发展,2022,59(7):1589−1609 doi: 10.7544/issn1000-1239.20210503 Li Xiaocan, Xie Kun, Zhang Dafang, et al. Survey of network anomaly detection based on low-rank decomposition[J]. Journal of Computer Research and Development, 2022, 59(7): 1589−1609 (in Chinese) doi: 10.7544/issn1000-1239.20210503
[5]	Noroozi M, Shah A. Towards optimal foreign object debris detection in an airport environment[J]. Expert Systems with Applications, 2023, 213(Part A): 118829
[6]	Jing Ying, Zheng Hong, Zheng Wentao, et al. A pixel-wise foreign object debris detection method based on multi-scale feature inpainting[J]. Aerospace, 2022, 9(9): 480 doi: 10.3390/aerospace9090480
[7]	杨帆,肖斌,於志文. 监控视频的异常检测与建模综述[J]. 计算机研究与发展,2021,58(12):2708−2723 doi: 10.7544/issn1000-1239.2021.20200638 Yang Fan, Xiao Bin, Yu Zhiwen. Anomaly detection and modeling of surveillance video[J]. Journal of Computer Research and Development, 2021, 58(12): 2708−2723 (in Chinese) doi: 10.7544/issn1000-1239.2021.20200638
[8]	Bergmann P, Batzner K, Fauser M, et al. The MVTec anomaly detection dataset: A comprehensive real-world dataset for unsupervised anomaly detection[J]. International Journal of Computer Vision, 2021, 129(4): 1038−1059 doi: 10.1007/s11263-020-01400-4
[9]	Munyer T, Brinkman D, Zhong Xin, et al. Foreign object debris detection for airport pavement images based on self-supervised localization and vision transformer[C]//Proc of the 9th Int Conf on Computational Science and Computational Intelligence. Piscataway, NJ: IEEE, 2022: 1388−1394
[10]	Liu Jialing, Lu Yifeng. A lightweight foreign object debris detection algorithm for airport runway[C]//Proc of the 5th Int Conf on Computer Science and Software Engineering. New York: ACM, 2022: 451−455
[11]	Duygu K. Examining the effect of different networks on foreign object debris detection[J]. Bitlis Eren Üniversitesi Fen Bilimleri Dergisi, 2023, 12(1): 151−157
[12]	Martí L, Sanchez-Pi N, Molina J M, et al. Anomaly detection based on sensor data in petroleum industry applications[J]. Sensors, 2015, 15(2): 2774−2797
[13]	Baur C, Wiestler B, Albarqouni S, et al. Deep autoencoding models for unsupervised anomaly segmentation in brain MR images[C]//Proc of the 4th Int Workshop on Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries. Berlin: Springer, 2018: 161−169
[14]	Zhao Chunhui, Yao Xifeng. Progressive line processing of global and local real-time anomaly detection in hyperspectral images[J]. Journal of Real-Time Image Processing, 2019, 16(6): 2289−2303 doi: 10.1007/s11554-017-0738-8
[15]	Wyatt J, Leach A, Schmon S M, et al. Anoddpm: Anomaly detection with denoising diffusion probabilistic models using simplex noise[C]//Proc of the 39th IEEE/CVF Conf on Computer Vision and Pattern Recognition (CVPR). Piscataway, NJ: IEEE, 2022: 650−656
[16]	Nawaratne R, Alahakoon D, De Silva D, et al. Spatiotemporal anomaly detection using deep learning for real-time video surveillance[J]. IEEE Transactions on Industrial Informatics, 2019, 16(1): 393−402
[17]	Chandola V, Banerjee A, Kumar V. Anomaly detection: A survey[J]. ACM Computing Surveys, 2009, 41(3): 1−58
[18]	Yang Jingkang, Zhou Kaiyang, Li Yixuan, et al. Generalized out-of-distribution detection: A survey[J]. arXiv preprint, arXiv: 2110.11334, 2021
[19]	Cohen N, Hoshen Y. Sub-image anomaly detection with deep pyramid correspondences[J]. arXiv preprint, arXiv: 2005.02357, 2020
[20]	Defard T, Setkov A, Loesch A, et al. Padim: A patch distribution modeling framework for anomaly detection and localization[C]//Proc of the 26th Int Conf on Pattern Recognition. Berlin: Springer, 2021: 475−489
[21]	Roth K, Pemula L, Zepeda J, et al. Towards total recall in industrial anomaly detection[C]//Proc of the 39th IEEE/CVF Conf on Computer Vision and Pattern Recognition. Piscataway, NJ: IEEE, 2022: 14318−14328
[22]	Deng Hanqiu, Li Xingyu. Anomaly detection via reverse distillation from one-class embedding[C]//Proc of the 39th IEEE/CVF Conf on Computer Vision and Pattern Recognition. Piscataway, NJ: IEEE, 2022: 9737−9746
[23]	Gudovskiy D, Ishizaka S, Kozuka K. Cflow-ad: Real-time unsupervised anomaly detection with localization via conditional normalizing flows[C]//Proc of the 22nd IEEE/CVF Winter Conf on Applications of Computer Vision. Piscataway, NJ: IEEE, 2022: 98−107
[24]	Li Chunliang, Sohn K, Yoon J, et al. Cutpaste: Self-supervised learning for anomaly detection and localization[C]//Proc of the 38th IEEE/CVF Conf on Computer Vision and Pattern Recognition. Piscataway, NJ: IEEE, 2021: 9664−9674
[25]	Deng Jia, Dong Wei, Socher R, et al. ImageNet: A large-scale hierarchical image database[C]//Proc of the 26th IEEE/CVF Conf on Computer Vision and Pattern Recognition. Piscataway, NJ: IEEE, 2009: 248−255
[26]	Akcay S, Atapour-Abarghouei A, Breckon T P. Ganomaly: Semi-supervised anomaly detection via adversarial training[C]//Proc of the 14th Asian Conf on Computer Vision. Berlin: Springer, 2019: 622−637
[27]	Gong Dong, Liu Lingqiao, Le V, et al. Memorizing normality to detect anomaly: Memory-augmented deep autoencoder for unsupervised anomaly detection[C]//Proc of the 17th IEEE/CVF Int Conf on Computer Vision. Piscataway, NJ: IEEE, 2019: 1705−1714
[28]	Dehaene D, Eline P. Anomaly localization by modeling perceptual features[J]. arXiv preprint, arXiv: 2008.05369, 2020
[29]	Song J W, Kong K, Park Y I, et al. AnoSeg: Anomaly segmentation network using self-supervised learning[J]. arXiv preprint, arXiv: 2110.03396, 2021
[30]	Hou Jinlei, Zhang Yingying, Zhong Qiaoyong, et al. Divide-and-assemble: Learning block-wise memory for unsupervised anomaly detection[C]//Proc of the 18th IEEE/CVF Int Conf on Computer Vision. Piscataway, NJ: IEEE, 2021: 8791−8800
[31]	Ristea N C, Madan N, Ionescu R T, et al. Self-supervised predictive convolutional attentive block for anomaly detection[C]//Proc of the 39th IEEE/CVF Conf on Computer Vision and Pattern Recognition. Piscataway, NJ: IEEE, 2022: 13576−13586
[32]	Zavrtanik V, Kristan M, Skočaj D. Draem-a discriminatively trained reconstruction embedding for surface anomaly detection[C]//Proc of the 18th IEEE/CVF Int Conf on Computer Vision. Piscataway, NJ: IEEE, 2021: 8330−8339
[33]	Yuan Jingyang, Li Bin, Xue Xiangyang. Generative modeling of infinite occluded objects for compositional scene representation[C]//Proc of the 36th Int Conf on Machine Learning. New York: ACM, 2019: 7222−7231
[34]	Locatello F, Weissenborn D, Unterthiner T, et al. Object-centric learning with slot attention[J]. Advances in Neural Information Processing Systems, 2020, 33: 11525−11538
[35]	Zou Y, Jeong J, Pemula L, et al. Spot-the-difference self-supervised pre-training for anomaly detection and segmentation[C]//Proc of the 17th European Conf on Computer Vision. Berlin: Springer, 2022: 392−408
[36]	Munyer T, Huang P C, Huang Chenyu, et al. FOD-A: A dataset for foreign object debris in airports[J]. arXiv preprint, arXiv: 2110.03072, 2021
[37]	Idrees H, Shah M, Surette R. Enhancing camera surveillance using computer vision: A research note[J]. Policing: An International Journal, 2018, 41(2): 292−307 doi: 10.1108/PIJPSM-11-2016-0158
[38]	Diehl C P, Hampshire J B. Real-time object classification and novelty detection for collaborative video surveillance[C]//Proc of the 12th Int Joint Conf on Neural Networks. Piscataway, NJ: IEEE, 2002, 3: 2620−2625
[39]	Bergmann P, Fauser M, Sattlegger D, et al. Uninformed students: Student-teacher anomaly detection with discriminative latent embeddings[C]//Proc of the 37th IEEE/CVF Conf on Computer Vision and Pattern Recognition. Piscataway, NJ: IEEE, 2020: 4183−4192
[40]	Salehi M, Sadjadi N, Baselizadeh S, et al. Multiresolution knowledge distillation for anomaly detection[C]//Proc of the 38th IEEE/CVF Conf on Computer Vision and Pattern Recognition. Piscataway, NJ: IEEE, 2021: 14902−14912
[41]	Wang Guodong, Han Shumin, Ding E, et al. Student-teacher feature pyramid matching for anomaly detection[J]. arXiv preprint, arXiv: 2103.04257, 2021
[42]	Dehaene D, Frigo O, Combrexelle S, et al. Iterative energy-based projection on a normal data manifold for anomaly localization[J]. arXiv preprint, arXiv: 2002.03734, 2020
[43]	Schlegl T, Seeböck P, Waldstein S M, et al. Unsupervised anomaly detection with generative adversarial networks to guide marker discovery[C]//Proc of the 25th Int Conf on Information Processing in Medical Imaging. Berlin: Springer, 2017: 146−157
[44]	Huang J, Murphy K. Efficient inference in occlusion-aware generative models of images[J]. arXiv preprint, arXiv: 1511.06362, 2015
[45]	Eslami S M, Heess N, Weber T, et al. Attend, infer, repeat: Fast scene understanding with generative models[J]. Advances in Neural Information Processing Systems, 2016, 29: 3225−3233
[46]	Crawford E, Pineau J. Spatially invariant unsupervised object detection with convolutional neural networks[C]//Pro of the 33rd AAAI Conf on Artificial Intelligence. Palo Alto, CA: AAAI, 2019: 3412−3420
[47]	Lin Zhixuan, Wu Y F, Peri S V, et al. Space: Unsupervised object-oriented scene representation via spatial attention and decomposition[J]. arXiv preprint, arXiv: 2001.02407, 2020
[48]	Greff K, Kaufman R L, Kabra R, et al. Multi-object representation learning with iterative variational inference[C]//Proc of the 36th Int Conf on Machine Learning. New York: ACM, 2019: 2424−2433
[49]	Engelcke M, Kosiorek A R, Jones O P, et al. Genesis: Generative scene inference and sampling with object-centric latent representations[J]. arXiv preprint, arXiv: 1907.13052, 2019
[50]	Jiang Jindong, Ahn S. Generative neurosymbolic machines[J]. Advances in Neural Information Processing Systems, 2020, 33: 12572−12582
[51]	Yuan Jingyang, Li Bin, Xue Xiangyang. Knowledge-guided object discovery with acquired deep impressions[C]//Proc of the 35th AAAI Conf on Artificial Intelligence. Palo Alto, CA: AAAI, 2021: 10798−10806
[52]	Ronneberger O, Fischer P, Brox T. U-net: Convolutional networks for biomedical image segmentation[C]//Proc of the 18th Int Conf Medical Image on Medical Image Computing and Computer-Assisted Intervention. Berlin: Springer, 2015: 234−241
[53]	Hofmeyr D P. Fast exact univariate kernel density estimation[J]. arXiv preprint, arXiv: 1806.00690, 2018
[54]	He Kaiming, Zhang Xiangyu, Ren Shaoqing, et al. Deep residual learning for image recognition[C]//Proc of the 33rd IEEE Conf on Computer Vision and Pattern Recognition. Piscataway, NJ: IEEE, 2016: 770−778

[1]	Li Kunze, Zhang Yu. Adaptive Pipeline Unsupervised Question Generation Method[J]. Journal of Computer Research and Development. DOI: 10.7544/issn1000-1239.202330857
[2]	Wei Jia, Zhang Xingjun, Wang Longxiang, Zhao Mingqiang, Dong Xiaoshe. MC² Energy Consumption Model for Massively Distributed Data Parallel Training of Deep Neural Network[J]. Journal of Computer Research and Development, 2024, 61(12): 2985-3004. DOI: 10.7544/issn1000-1239.202330164
[3]	Tang Xiaolan, Liang Yuting, Chen Wenlong. Multi-Stage Federated Learning Mechanism with non-IID Data in Internet of Vehicles[J]. Journal of Computer Research and Development, 2024, 61(9): 2170-2184. DOI: 10.7544/issn1000-1239.202330885
[4]	Zhang Naizhou, Cao Wei, Zhang Xiaojian, Li Shijun. Conversation Generation Based on Variational Attention Knowledge Selection and Pre-trained Language Model[J]. Journal of Computer Research and Development. DOI: 10.7544/issn1000-1239.202440551
[5]	Wang Yan, Tong Xiangrong. Cross-Domain Trust Prediction Based on tri-training and Extreme Learning Machine[J]. Journal of Computer Research and Development, 2022, 59(9): 2015-2026. DOI: 10.7544/issn1000-1239.20210467
[6]	Zhang Yong, Chen Rongrong, Zhang Jing. Safe Tri-training Algorithm Based on Cross Entropy[J]. Journal of Computer Research and Development, 2021, 58(1): 60-69. DOI: 10.7544/issn1000-1239.2021.20190838
[7]	Zhang Heng, Zhang Libo, WuYanjun. Large-Scale Graph Processing on Multi-GPU Platforms[J]. Journal of Computer Research and Development, 2018, 55(2): 273-288. DOI: 10.7544/issn1000-1239.2018.20170697
[8]	Shao Zengzhen, Wang Hongguo, Liu Hong, Song Chaochao, Meng Chunhua, Yu Hongling. Heuristic Optimization Algorithms of Multi-Carpooling Problem Based on Two-Stage Clustering[J]. Journal of Computer Research and Development, 2013, 50(11): 2325-2335.
[9]	Tang Huanling, Lin Zhengkui, Lu Mingyu, Wu Jun. An Advanced Co-Training Algorithm Based on Mutual Independence and Diversity Measures[J]. Journal of Computer Research and Development, 2008, 45(11): 1874-1881.
[10]	Wang Liming, Huang Houkuan. A Multistage-Based Framework for Multi-Agent Multi-Issue Negotiation[J]. Journal of Computer Research and Development, 2005, 42(11): 1849-1855.