Abstract: Anomaly detection aims to detect abnormal samples among many normal samples. In the era of big data, how to apply anomaly detection to real-world scenarios has become one of the most critical problems to consider. Currently, the existing models can hardly cope with dynamic interference such as occlusion, lighting, and color difference in real-world scenarios and cannot quickly migrate application scenarios. This paper proposes a deep learning model based on Background-Foreground Modeling for anomaly detection tasks. Our model first reconstructs the input image into a clean background image without abnormal objects through the feature extraction network and preserves the possible dynamic interference of the image through skip-connection. After obtaining the reconstructed background, this model extracts the position information of abnormal objects through the Spatial Transformation Network, uses an autoencoder to extract latent space representations of the appearance, shape, and presence information of abnormal objects, and reconstructs them. Finally, this model combines the reconstructed abnormal objects and the background image to obtain an overall reconstructed image and realizes anomaly detection by setting a threshold for the presence information of abnormal objects. To validate the effectiveness of the method, this paper collects data from a real circuit board assembly environment and simulates a scenario with limited annotations in actual production, resulting in the creation of a Foreign Object in Circuit Board (FO-CB) Dataset for analysis. Additionally, this paper also conducts experimental validation on the Foreign Object Debris in Airport (FOD-A) Dataset. The experimental results show that our proposed method performs well on the synthetic dataset and detects all anomalous objects in 9 actual scene data, with a miss rate of down to 0%, and can be applied to real-world circuit board assembly scenarios.