Survey of Large-Language-Model-Based Automated Program Repair

Xu Pengyu; Kuang Boyu; Su Mang; Fu Anmin

doi:10.7544/issn1000-1239.202440467

Journal of Computer Research and Development > 2024 > Accepted Manuscript > DOI: 10.7544/issn1000-1239.202440467

Xu Pengyu, Kuang Boyu, Su Mang, Fu Anmin. Survey of Large-Language-Model-Based Automated Program Repair[J]. Journal of Computer Research and Development. DOI: 10.7544/issn1000-1239.202440467

Citation:

Xu Pengyu, Kuang Boyu, Su Mang, Fu Anmin. Survey of Large-Language-Model-Based Automated Program Repair[J]. Journal of Computer Research and Development. DOI: 10.7544/issn1000-1239.202440467

Citation:

Xu Pengyu, Kuang Boyu, Su Mang, Fu Anmin. Survey of Large-Language-Model-Based Automated Program Repair[J]. Journal of Computer Research and Development. DOI: 10.7544/issn1000-1239.202440467

PDF (1285 KB)

Survey of Large-Language-Model-Based Automated Program Repair

Xu Pengyu^1,,
Kuang Boyu²,
Su Mang¹,
Fu Anmin^{1, 2, ,}

1.
School of Computer Science and Engineering, Nanjing University of Science and Technology, Nanjing 210094
2.
School of Cyber Science and Engineering, Nanjing University of Science and Technology, Nanjing 210094

Funds: This work was supported by the National Natural Science Foundation of China (62072239, 62372236), the Natural Science Foundation of Jiangsu Province (BK20211192), the Qing Lan Project of Jiangsu Province, the Jiangsu Funding Program for Excellent Postdoctoral Talent, and the Postdoctoral Fellowship Program of CPSF (GZB20240982).

More Information

Author Bio:
Xu Pengyu: born in 1999. Master candidate. His main research interests include large language model and automated program repair

Kuang Boyu: born in 1994. PhD. His main research interests include IoT security and IoV security

Su Mang: born in 1987. PhD, associate professor. Her main research interests include cloud security, access control, and right management

Fu Anmin: born in 1981. PhD, professor, PhD supervisor. Senior member of CCF. His main research interests include cryptography and privacy preserving
Received Date: June 04, 2024
Revised Date: October 21, 2024
Accepted Date: November 12, 2024
Available Online: November 14, 2024

Graphical Abstract

Abstract

Abstract

Software systems play an indispensable role across various industries, handling large-scale and high-density data. However, the numerous defects within these systems have troubled developers for a long time, constantly threatening the security of data elements. Automated Program Repair (APR) technology aims to assist developers in automatically fixing defects in code during software development process, thereby saving costs in software system development and maintenance, enhancing the confidentiality, availability, and integrity of data elements within software systems. With the development of Large Language Model (LLM) technology, many powerful code large language models have emerged. These models have demonstrated strong repair capabilities in the APR field, while also addressing shortcomings of traditional approaches in code comprehension and patch generation capabilities, further elevating the level of program repair tools. We thoroughly survey high-quality papers related to APR in recent years, summarizing the latest developments in the field. We then systematically categorize two types of LLM-based APR techniques: cloze style and neural machine translation style. We also conduct an in-depth comparison from various perspectives such as model usage, model size, types of defects repaired, programming languages involved, and the pros and cons of repair approaches. Additionally, we discuss the widely adopted APR datasets and metrics, and outline existing empirical studies. Finally, we summarize current challenges in the APR field along with future research directions.
- automated program repair,
- large language model,
- code large language model,
- software defect,
- patch generation

FullText(HTML)

References (80)

References

[1]	Wei Yuxiang, Xia Chunqiu, Zhang Lingming. Copiloting the copilots: Fusing large language models with completion engines for automated program repair [C] //Proc of the 31st ACM Joint Meeting on European Software Engineering Conf and Symp on the Foundations of Software Engineering. New York: ACM, 2023: 172−184
[2]	Huang Kai, Meng Xiangxin, Zhang Jian, et al. An empirical study on fine-tuning large language models of code for automated program repair [C] //Proc of the 38th IEEE/ACM Int Conf on Automated Software Engineering. Piscataway, NJ: IEEE, 2023: 1162−1174
[3]	田笑,常继友,张弛,等. 开源软件缺陷预测方法综述[J]. 计算机研究与发展,2023,60(7):1467−1488 doi: 10.7544/issn1000-1239.202221046 Tian Xiao, Chang Jiyou, Zhang Chi, et al. Survey of open-source software defect prediction method[J]. Journal of Computer Research and Development, 2023, 60(7): 1467−1488(in Chinese) doi: 10.7544/issn1000-1239.202221046
[4]	Gazzola L, Micucci D, Mariani L. Automatic software repair: A survey[J]. IEEE Transactions on Software Engineering, 2019, 45(1): 34−67 doi: 10.1109/TSE.2017.2755013
[5]	Zhang Quanjun, Fang Chunrong, Ma Yuxiang, et al. A survey of learning-based automated program repair[J]. ACM Transactions on Software Engineering and Methodology, 2024, 33(2): 1−69
[6]	姜佳君,陈俊洁,熊英飞. 软件缺陷自动修复技术综述[J]. 软件学报,2021,32(9):2665−2690 Jiang Jiajun, Chen Junjie, Xiong Yingfei. Survey of automatic program repair techniques[J]. Journal of Software, 2021, 32(9): 2665−2690(in Chinese)
[7]	王赞,郜健,陈翔,等. 自动程序修复方法研究述评[J]. 计算机学报,2018,41(3):588−610 doi: 10.11897/SP.J.1016.2018.00588 Wang Zan, Gao Jian, Chen Xiang, et al. Automatic program repair techniques: A survey[J]. Chinese Journal of Computers, 2018, 41(3): 588−610(in Chinese) doi: 10.11897/SP.J.1016.2018.00588
[8]	徐同同,刘逵,夏鑫. 漏洞自动修复研究综述[J]. 软件学报,2024,35(1):136−158 Xu Tongtong, Liu Kui, Xia Xin. Survey on automated vulnerability repair[J]. Journal of Software, 2024, 35(1): 136−158(in Chinese)
[9]	Fu M, Tantithamthavorn C, Le T, et al. VulRepair: A t5-based automated software vulnerability repair [C] //Proc of the 30th ACM Joint Meeting on European Software Engineering Conf and Symp on the Foundations of Software Engineering. New York: ACM, 2022: 935−947
[10]	彭长根,何兴,谭伟杰,等. 人工智能算法安全研究现状与对策[J]. 贵州师范大学学报:自然科学版,2022,40(6):1−16 Peng Changgen, He Xing, Tan Weijie, et al. Research status and countermeasures of artificial intelligence algorithm security[J]. Journal of Guizhou Normal University: Natural Sciences, 2022, 40(6): 1−16(in Chinese)
[11]	Naveed H, Khan A U, Qiu Shi, et al. A comprehensive overview of large language models [J]. arXiv preprint, arXiv: 2307.06435, 2023
[12]	Zheng Zibin, Ning Kaiwen, Wang Yanlin, et al. A survey of large language models for code: Evolution, benchmarking, and future trends [J]. arXiv preprint, arXiv: 2311.10372, 2023
[13]	Iyer S, Konstas I, Cheung A, et al. Summarizing source code using a neural attention model [C] //Proc of the 54th Annual Meeting of the Association for Computational Linguistics. Stroudsburg, PA: ACL, 2016: 2073–2083
[14]	Svyatkovskiy A, Deng Shaokun, Fu Shengyu, et al. Intellicode compose: Code generation using transformer [C] //Proc of the 28th ACM Joint Meeting on European Software Engineering Conf and Symp on the Foundations of Software Engineering. New York: ACM, 2020: 1433–1443
[15]	Raffel C, Shazeer N, Roberts A, et al. Exploring the limits of transfer learning with a unified text-to-text transformer[J]. Journal of Machine Learning Research, 2020, 21(1): 5485−5551
[16]	Devlin J, Chang Mingwei , Lee K, et al. BERT: Pre-training of deep bidirectional transformers for language understanding [C] //Proc of the Conf of the North American Chapter of the Association for Computational Linguistics. Stroudsburg, PA: ACL, 2019: 4171–4186
[17]	Feng Zhangyin, Guo Daya, Tang Duyu, et al. CodeBERT: A pre-trained model for programming and natural languages [C] //Proc of the Conf on Empirical Methods in Natural Language Processing. Stroudsburg, PA: ACL, 2020: 1536–1547
[18]	Wang Yue, Le H, Gotmare A, et al. CodeT5+: Open code large language models for code understanding and generation [C] //Proc of the Conf on Empirical Methods in Natural Language Processing. Stroudsburg, PA: ACL, 2024: 1069–1088
[19]	Wu Yi, Jiang Nan, Pham H V, et al. How effective are neural networks for fixing security vulnerabilities [C] //Proc of the 32nd ACM Int Symp on Software Testing and Analysis. New York: ACM, 2023: 1282–1294
[20]	Xia Chunqiu, Zhang Lingming. Less training, More repairing please: Revisiting automated program repair via zero-shot learning [C] //Proc of the 30th ACM Joint Meeting on European Software Engineering Conf and Symp on the Foundations of Software Engineering. New York: ACM, 2022: 959−971
[21]	Yang Deheng, Mao Xiaoguang, Chen Liqian, et al. TransplantFix: Graph differencing-based code transplantation for automated program repair [C/OL] //Proc of the 37th IEEE/ACM Int Conf on Automated Software Engineering. Piscataway, NJ: IEEE, 2022[2024-07-12]. https://conf.researchr.org/details/ase-2022/ase-2022-research-papers/1/TransplantFix-Graph-Differencing-based-Code-Transplantation-for-Automated-Program-Re
[22]	Wong C, Santiesteban P, Kästner C, et al. VarFix: Balancing edit expressiveness and search effectiveness in automated program repair [C] //Proc of the 29th ACM Joint Meeting on European Software Engineering Conf and Symp on the Foundations of Software Engineering. New York: ACM, 2021: 354−366
[23]	Lou Yiling, Chen Junjie, Zhang Lingming, et al. History-driven build failure fixing: How far are we [C] //Proc of the 28th ACM Int Symp on Software Testing and Analysis. New York: ACM, 2019: 43−54
[24]	Abdessalem R B, Panichella A, Nejati S, et al. Automated repair of feature interaction failures in automated driving systems [C] //Proc of the 29th ACM Int Symp on Software Testing and Analysis. New York: ACM, 2020: 88−100
[25]	Oh W, Oh H. PyTER: Effective program repair for python type errors [C] //Proc of the 30th ACM Joint Meeting on European Software Engineering Conf and Symp on the Foundations of Software Engineering. New York: ACM, 2022: 922−934
[26]	Zheng Guolong, Nguyen T, Brida S G, et al. ATR: Template-based repair for alloy specifications [C] //Proc of the 31st ACM Int Symp on Software Testing and Analysis. New York: ACM, 2022: 666−677
[27]	Koyuncu A, Liu Kui, Bissyandé T F, et al. iFixR: Bug report driven program repair [C] //Proc of the 27th ACM Joint Meeting on European Software Engineering Conf and Symp on the Foundations of Software Engineering. New York: ACM, 2019: 314−325
[28]	Liu Kui, Koyuncu A, Bissyandé T F, et al. You cannot fix what you cannot find! An investigation of fault localization bias in benchmarking automated program repair systems [C] //Proc of the 12th IEEE Int Conf on Software Testing, Verification and Validation. Piscataway, NJ: IEEE, 2019: 102−113
[29]	Liu Kui, Koyuncu A, Kim D, et al. TBar: Revisiting template-based automated program repair [C] //Proc of the 28th ACM Int Symp on Software Testing and Analysis. New York: ACM, 2019: 31−42
[30]	Ghanbari A, Benton S, Zhang Lingming. Practical program repair via bytecode mutation [C] //Proc of the 28th ACM Int Symp on Software Testing and Analysis. New York: ACM, 2019: 19−30
[31]	Afzal A, Motwani M, Stolee K T, et al. SOSRepair: Expressive semantic search for real-world program repair[J]. IEEE Transactions on Software Engineering, 2021, 47(10): 2162−2181 doi: 10.1109/TSE.2019.2944914
[32]	牛翠翠,潘正芝,刘海. 差分隐私生成式对抗网络的框架与方法综述[J]. 贵州师范大学学报:自然科学版,2022,40(4):84−99 Niu Cuicui, Pan Zhengzhi, Liu Hai. Framework and approaches of differentially private generative adversarial networks: A survey[J]. Journal of Guizhou Normal University: Natural Sciences, 2022, 40(4): 84−99(in Chinese)
[33]	Bavishi R, Yoshida H, Prasad M R. Phoenix: Automated data-driven synthesis of repairs for static analysis violations [C] //Proc of the 27th ACM Joint Meeting on European Software Engineering Conf and Symp on the Foundations of Software Engineering. New York: ACM, 2019: 613−624
[34]	Li Yi, Wang Shaohua, Nguyen T N. Dear: A novel deep learning-based approach for automated program repair [C] //Proc of the 44th ACM/IEEE Int Conf on Software Engineering. New York: ACM, 2022: 511−523
[35]	Lutellier T, Pham H V, Pang L, et al. Coconut: Combining context-aware neural translation models using ensemble for program repair [C] //Proc of the 29th ACM Int Symp on Software Testing and Analysis. New York: ACM, 2020: 101−114
[36]	曹鹤玲,刘昱,韩栋. 基于自注意力机制神经机器翻译的软件缺陷自动修复方法[J]. 电子学报,2024,52(3):945−956 Cao Heling, Liu Yu, Han Dong. Self-attention neural machine translation for automatic software repair[J]. Acta Electronica Sinica, 2024, 52(3): 945−956(in Chinese)
[37]	Xia Chunqiu, Wei Yuxiang, Zhang Lingming. Automated program repair in the era of large pre-trained language models [C] //Proc of the 45th ACM/IEEE Int Conf on Software Engineering. New York: ACM, 2023: 1482–1494
[38]	Jiang Nan, Lutellier T, Tan Lin. Cure: Code-aware neural machine translation for automatic program repair [C] //Proc of the 43rd ACM/IEEE Int Conf on Software Engineering. New York: ACM, 2021: 1161−1173
[39]	Prenner J A, Robbes R. Out of context: How important is local context in neural program repair [C/OL] //Proc of the 46th ACM/IEEE Int Conf on Software Engineering. New York: ACM, 2024[2024-07-12]. https://conf.researchr.org/details/icse-2024/icse-2024-research-track/82/Out-of-Context-How-important-is-Local-Context-in-Neural-Program-Repair-
[40]	Zhang Quanjun, Fang Chunrong, Zhang Tongke, et al. Gamma: Revisiting template-based automated program repair via mask prediction [C] //Proc of the 38th IEEE/ACM Int Conf on Automated Software Engineering. Piscataway, NJ: IEEE, 2023: 535−547
[41]	Fan Zhiyu, Gao Xiang, Mirchev M, et al. Automated repair of programs from large language models [C] //Proc of the 45th ACM/IEEE Int Conf on Software Engineering. New York: ACM, 2023: 1469−1481
[42]	Peng Yun, Gao Shuzheng, Gao Cuiyun, et al. Domain knowledge matters: Improving prompts with fix templates for repairing python type errors [C/OL] //Proc of the 46th ACM/IEEE Int Conf on Software Engineering. New York: ACM, 2024[2024-07-12]. https://conf.researchr.org/details/icse-2024/icse-2024-research-track/8/Domain-Knowledge-Matters-Improving-Prompts-with-Fix-Templates-for-Repairing-Python-T
[43]	Liu Kui, Koyuncu A, Kim D, et al. Avatar: Fixing semantic bugs with fix patterns of static analysis violations [C] //Proc of the 26th IEEE Int Conf on Software Analysis, Evolution and Reengineering. Piscataway, NJ: IEEE, 2019: 456–467
[44]	Liu Xuliang, Zhong Hao. Mining stackoverflow for program repair [C] //Proc of the 25th IEEE Int Conf on Software Analysis, Evolution and Reengineering. Piscataway, NJ: IEEE, 2018: 118–129
[45]	Koyuncu A, Liu Kui, Bissyande T F, et al. FixMiner: Mining relevant fix patterns for automated program repair[J]. Empirical Software Engineering, 2020, 25(3): 1980−2024 doi: 10.1007/s10664-019-09780-z
[46]	Liu Kui, Kim D, Bissyande T F, et al. Mining fix patterns for findbugs violations[J]. IEEE Trans on Software Engineering, 2018, 47(1): 165−188
[47]	Guo Daya, Lu Shuai, Duan Nan, et al. UniXcoder: Unified cross-modal pre-training for code representation [C] //Proc of the 60th Annual Meeting of the Association for Computational Linguistics. Stroudsburg, PA: ACL, 2022: 7212−7225
[48]	Wang Yue, Wang Weishi, Joty S, et al. CodeT5: Identifier-aware unified pre-trained encoder-decoder models for code understanding and generation [C] //Proc of the Conf on Empirical Methods in Natural Language Processing. Stroudsburg, PA: ACL, 2021: 8696–8708
[49]	Meng Yu, Krishnan J, Wang Sinong, et al. Representation deficiency in masked language modeling [C/OL] //Proc of the 12th Int Conf on Learning Representations, 2024[2024-07-12]. https://iclr.cc/virtual/2024/poster/18345
[50]	Jiang Nan, Liu K, Lutellier T, et al. Impact of code language models on automated program repair [C] //Proc of the 45th ACM/IEEE Int Conf on Software Engineering. New York: ACM, 2023: 1430−1442
[51]	Zhou Xin, Kim K, Xu Bowen, et al. Out of sight, Out of mind: Better automatic vulnerability repair by broadening input ranges and sources [C/OL] //Proc of the 46th ACM/IEEE Int Conf on Software Engineering. New York: ACM, 2024[2024-07-12]. https://conf.researchr.org/details/icse-2024/icse-2024-research-track/222/Out-of-Sight-Out-of-Mind-Better-Automatic-Vulnerability-Repair-by-Broadening-Input-
[52]	Yuan Wei, Zhang Quanjun, He T, et al. CIRCLE: Continual repair across programming languages [C] //Proc of the 31st ACM Int Symp on Software Testing and Analysis. New York: ACM, 2022: 678−690
[53]	Ye He, Martinez M, Monperrus M. Neural program repair with execution-based backpropagation [C] //Proc of the 44th ACM/IEEE Int Conf on Software Engineering. New York: ACM, 2022: 1506−1518
[54]	Ye He, Martinez M, Luo Xiapu, et al. SelfAPR: Self-supervised program repair with test execution diagnostics [C/OL] //Proc of the 37th IEEE/ACM Int Conf on Automated Software Engineering. Piscataway, NJ: IEEE, 2022[2024-07-12]. https://conf.researchr.org/details/ase-2022/ase-2022-research-papers/45/SelfAPR-Self-supervised-Program-Repair-with-Test-Execution-Diagnostics
[55]	Wang Weishi, Wang Yue, Joty S, et al. RAP-Gen: Retrieval-augmented patch generation with codet5 for automatic program repair [C] //Proc of the 31st ACM Joint Meeting on European Software Engineering Conf and Symp on the Foundations of Software Engineering. New York: ACM, 2023: 146−158
[56]	Just R, Jalali D, Ernst M D. Defects4J: A database of existing faults to enable controlled testing studies for Java programs [C] //Proc of the 23rd ACM Int Symp on Software Testing and Analysis. New York: ACM, 2014: 437–440
[57]	Zhu Qihao, Sun Zeyu, Xiao Yuanan, et al. A syntax-guided edit decoder for neural program repair [C] //Proc of the 29th ACM Joint Meeting on European Software Engineering Conf and Symp on the Foundations of Software Engineering. New York: ACM, 2021: 341−353
[58]	Tufano M, Watson C, Bavota G, et al. An empirical study on learning bug-fixing patches in the wild via neural machine translation[J]. ACM Trans on Software Engineering and Methodology, 2019, 28(4): 1−29
[59]	Lin D, Koppel J, Chen A, et al. QuixBugs: A multi-lingual program repair benchmark set based on the quixey challenge [C] //Proc Companion of the ACM SIGPLAN Int Conf on Systems, Programming, Languages, and Applications: Software for Humanity. New York: ACM, 2017: 55–56
[60]	Chen Zimin, Kommrusch S, Tufano M, et al. SequenceR: Sequence-to-sequence learning for end-to-end program repair[J]. ACM Trans on Software Engineering and Methodology, 2021, 47(9): 1943−1959
[61]	Berabi B, He Jingxuan, Raychev V, et al. TFix: Learning to fix coding errors with a text-to-text transformer [C] //Proc of the 20th Int Conf on Machine Learning. New York: ACM, 2021: 780–791
[62]	Fan Jiahao, Li Yi, Wang Shaohua, et al. A C/C++ code vulnerability dataset with code changes and CVE summaries [C] //Proc of the 17th IEEE/ACM Int Conf on Mining Software Repositories. Piscataway, NJ: IEEE, 2020: 508−512
[63]	Bhandari G, Naseer A, Moonen L. CVEfixes: Automated collection of vulnerabilities and their fixes from open-source software [C] //Proc of the 17th Int Conf on Predictive Models and Data Analytics in Software Engineering. New York: ACM, 2021: 30−39
[64]	Goues C L, Holtschulte N, Smith E K, et al. The manybugs and introclass benchmarks for automated repair of c programs[J]. IEEE Transactions on Software Engineering, 2015, 41(12): 1236−1256 doi: 10.1109/TSE.2015.2454513
[65]	Bui Q, Scandariato R, Ferreyra N E D. Vul4J: A dataset of reproducible java vulnerabilities geared towards the study of program repair techniques [C] //Proc of the 19th IEEE/ACM Int Conf on Mining Software Repositories. Piscataway, NJ: IEEE, 2022: 464–468
[66]	Hanam Q, Brito F S M, Mesbah A. Discovering bug patterns in javascript [C] //Proc of the 24th ACM SIGSOFT Int Symp on Foundations of Software Engineering. New York: ACM, 2016: 144–156
[67]	Saha R K, Lyu Y, Lam W, et al. Bugs. Jar: A large-scale, Diverse dataset of real-world java bugs [C] //Proc of the 15th IEEE/ACM Int Conf on Mining Software Repositories. Piscataway, NJ: IEEE, 2018: 10–13
[68]	Monperrus M, Martinez M, Ye He, et al. Megadiff: A dataset of 600k Java source code changes categorized by diff size [J]. arXiv preprint, arXiv: 2108.04631, 2021
[69]	Madeiral F, Urli S, Maia M, et al. BEARS: An extensible java bug benchmark for automatic program repair studies [C] //Proc of the 26th IEEE Int Conf on Software Analysis, Evolution and Reengineering. Piscataway, NJ: IEEE, 2019: 468−478
[70]	Chen Zimin, Monperrus M. The codrep machine learning on source code competition [J]. arXiv preprint, arXiv: 1807.03200, 2018
[71]	Richter C, Wehrheim H. TSSB−3M: Mining single statement bugs at massive scale [C] //Proc of the 19th IEEE/ACM Int Conf on Mining Software Repositories. Piscataway, NJ: IEEE, 2022: 418–422
[72]	Karampatsis R, Sutton C. How often do single statement bugs occur? The manysstubs4j dataset [C] //Proc of the 17th IEEE/ACM Int Conf on Mining Software Repositories. Piscataway, NJ: IEEE, 2020: 573–577
[73]	Guo Daya, Ren Shuo, Lu Shuai, et al. GraphCodeBERT: Pre-training code representations with data flow [C/OL] //Proc of the 9th Int Conf on Learning Representations. 2021[2024-07-12]. https://iclr.cc/virtual/2021/poster/2598
[74]	Gong Linyuan, Elhoushi M, Cheung A. AST-T5: Structure-aware pretraining for code generation and understanding [J]. arXiv preprint, arXiv: 2401.03003, 2024
[75]	Shen Bo, Zhang Jiaxin, Chen Taihong, et al. Pangu-coder2: Boosting large language models for code with ranking feedback [J]. arXiv preprint, arXiv: 2307.14936, 2023
[76]	Di Peng, Li Jianguo, Yu Hang, et al. Codefuse−13b: A pretrained multi-lingual code large language model [C] //Proc of the 46th Int Conf on Software Engineering: Software Engineering in Practice. Piscataway, NJ: IEEE, 2024: 418−429
[77]	Ye He, Monperrus M. ITER: Iterative neural repair for multi-location patches [C/OL]//Proc of the 46th ACM/IEEE Int Conf on Software Engineering. New York: ACM, 2024[2024-07-12]. https://conf.researchr.org/details/icse-2024/icse-2024-research-track/55/ITER-Iterative-Neural-Repair-for-Multi-Location-Patches
[78]	梅瑞,严寒冰,沈元,等. 二进制代码切片技术在恶意代码检测中的应用研究[J]. 信息安全学报,2021,6(3):125−140 Mei Rui, Yan Hanbing, Shen Yuan, et al. Application research of slicing technology of binary executables in malware detection[J]. Journal of Cyber Security, 2021, 6(3): 125−140(in Chinese)
[79]	张健,张超,玄跻峰,等. 程序分析研究进展[J]. 软件学报,2019,30(1):80−109 Zhang Jian, Zhang Chao, Xuan Jifeng, et al. Recent progress in program analysis[J]. Journal of Software, 2019, 30(1): 80−109(in Chinese)
[80]	谢肖飞,李晓红,陈翔,等. 基于符号执行与模糊测试的混合测试方法[J]. 软件学报,2019,30(10):3071−3089 Xie Xiaofei, Li Xiaohong, Chen Xiang, et al. Hybrid testing based on symbolic execution and fuzzing[J]. Journal of Software, 2019, 30(10): 3071−3089(in Chinese)

[1]	Li Dongwen, Zhong Zhenyu, Sun Yufei, Shen Junyu, Ma Zizhi, Yu Chuanyue, Zhang Yuzhi. LingLong: A High-Quality Small-Scale Chinese Pre-trained Language Model[J]. Journal of Computer Research and Development, 2025, 62(3): 682-693. DOI: 10.7544/issn1000-1239.202330844
[2]	Jiang Yi, Yang Yong, Yin Jiali, Liu Xiaolei, Li Jiliang, Wang Wei, Tian Youliang, Wu Yingcai, Ji Shouling. A Survey on Security and Privacy Risks in Large Language Models[J]. Journal of Computer Research and Development. DOI: 10.7544/issn1000-1239.202440265
[3]	Zhang Naizhou, Cao Wei, Zhang Xiaojian, Li Shijun. Conversation Generation Based on Variational Attention Knowledge Selection and Pre-trained Language Model[J]. Journal of Computer Research and Development. DOI: 10.7544/issn1000-1239.202440551
[4]	Yi Xiaoyuan, Xie Xing. Unpacking the Ethical Value Alignment in Big Models[J]. Journal of Computer Research and Development, 2023, 60(9): 1926-1945. DOI: 10.7544/issn1000-1239.202330553
[5]	Feng Jun, Shi Yichen, Gao Yuhao, He Jingjing, Yu Zitong. Domain Adaptation for Face Anti-Spoofing Based on Dual Disentanglement and Liveness Feature Progressive Alignment[J]. Journal of Computer Research and Development, 2023, 60(8): 1727-1739. DOI: 10.7544/issn1000-1239.202330251
[6]	Zeng Weixin, Zhao Xiang, Tang Jiuyang, Tan Zhen, Wang Wei. Iterative Entity Alignment via Re-Ranking[J]. Journal of Computer Research and Development, 2020, 57(7): 1460-1471. DOI: 10.7544/issn1000-1239.2020.20190643
[7]	Shi Haihe, Zhou Weixing. Design and Implementation of Pairwise Sequence Alignment Algorithm Components Based on Dynamic Programming[J]. Journal of Computer Research and Development, 2019, 56(9): 1907-1917. DOI: 10.7544/issn1000-1239.2019.20180835
[8]	Jia Xibin, Jin Ya, Chen Juncheng. Domain Alignment Based on Multi-Viewpoint Domain-Shared Feature for Cross-Domain Sentiment Classification[J]. Journal of Computer Research and Development, 2018, 55(11): 2439-2451. DOI: 10.7544/issn1000-1239.2018.20170496
[9]	Wang Yuquan, Wen Lijie, Yan Zhiqiang. Alignment Based Conformance Checking Algorithm for BPMN 2.0 Model[J]. Journal of Computer Research and Development, 2017, 54(9): 1920-1930. DOI: 10.7544/issn1000-1239.2017.20160756
[10]	Zhuang Yan, Li Guoliang, Feng Jianhua. A Survey on Entity Alignment of Knowledge Base[J]. Journal of Computer Research and Development, 2016, 53(1): 165-192. DOI: 10.7544/issn1000-1239.2016.20150661