Simulation-Based Fuzzing for Autonomous Driving Systems: Landscapes, Challenges and Prospects

Dai Jiarun; Li Zhongrui; Zhang Wanqi; Zhang Yuan; Yang Min

doi:10.7544/issn1000-1239.202330156

Journal of Computer Research and Development > 2023 > 60(7): 1433-1447. > DOI: 10.7544/issn1000-1239.202330156 CSTR: 32373.14.issn1000-1239.202330156

Dai Jiarun, Li Zhongrui, Zhang Wanqi, Zhang Yuan, Yang Min. Simulation-Based Fuzzing for Autonomous Driving Systems: Landscapes, Challenges and Prospects[J]. Journal of Computer Research and Development, 2023, 60(7): 1433-1447. DOI: 10.7544/issn1000-1239.202330156

Citation:

PDF (1560 KB)

Simulation-Based Fuzzing for Autonomous Driving Systems: Landscapes, Challenges and Prospects

College of Computer Science and Technology, Fudan University, Shanghai 200438

Funds: This work was supported by the National Key Research and Development Program (2021YFB3101200), the National Natural Science Foundation of China (62172105), the Shanghai Rising-Star Program (21QA1400700), and the Shanghai Pilot Program for Basic Research (21TQ1400100:21TQ012).

More Information

Author Bio:
Dai Jiarun: born in 1996. PhD candidate. His main research interests include program analysis and safety assessment of autonomous driving systems

Li Zhongrui: born in 1998. Master candidate. His main research interests include vulnerability detection and program analysis

Zhang Wanqi: born in 1992. Research assistant. Her main research interest includes intelligent system security

Zhang Yuan: born in 1987. PhD, associate professor, PhD supervisor. Member of CCF. His main research interests include software security and program analysis

Yang Min: born in 1979. PhD, professor, PhD supervisor. Member of CCF. His main research interests include system security and AI security
Received Date: March 27, 2023
Revised Date: April 27, 2023
Available Online: June 14, 2023

Graphical Abstract

Abstract

Abstract

Autonomous driving has become one of the most revolutionary innovations in the field of transportation. Its extensive application prospect drives many manufacturers to develop and deploy autonomous vehicles on public roads. However, given the fact that traffic accidents involving autonomous vehicles continue to occur, safety has become the main obstacle for their widespread adoption. To tackle this issue, simulation-based fuzzing is gaining attention due to its capability to automatically identify flaws of autonomous driving systems which may cause traffic accidents. However, this technology is still in its early stages of research, and existing work are far from faithfully addressing the potential safety issues of autonomous driving systems. Considering this, we first introduce the basic architecture and research status of this technology. After that, we summarize the shortcomings of existing work, as well as the challenges to achieve effective simulation-based fuzzing. Accordingly, we try to propose solutions which can potentially tackle these challenges. To showcase the effectiveness of these solutions, we apply them in the safety assessment of popular open-source autonomous driving systems (i.e., Apollo and Autoware). Results show that, these solutions can boost the capability of existing simulation-based fuzzers in identifying and diagnosing safety-related flaws of Apollo and Autoware. Finally, we try to pinpoint the future research directions, so as to ease follow-up research.
- autonomous driving system (ADS),
- functional,
- simulation,
- fuzzing,
- simulation-based fuzzing

FullText(HTML)

References (68)

References

[1]	California DMV. California DMV[EB/OL]. [2023-02-27].https://www.dmv.ca.gov/portal/
[2]	California DMV. Autonomous Vehicle Collision Reports[EB/OL]. [2023-02-27]. https://www.dmv.ca.gov/portal/vehicle-industry-services/autonomous-vehicles/autonomous-vehicle-collision-reports/
[3]	ISO. ISO 21448: 2022[EB/OL]. [2023-03-07]. https://www.iso.org/standard/77490.html
[4]	全国标准信息公共服务平台. 道路车辆预期功能安全[EB/OL]. [2023-03-07]. https://std.samr.gov.cn/gb/search/gbDetailed?id=E116673EBDEFA3B7E05053BE0A0AC6BF National Public Service Platform for Standards Information. Road Vehicles-Safety of the Intended Functionality[EB/OL]. [2023-03-07].https://std.samr.gov.cn/gb/search/gbDetailed?id=E116673EBDEFA3B7E05397BE0A0AC6BF
[5]	Sportillo D, Paljic A, Ojeda L. On-road evaluation of autonomous driving training[C]//Proc of the 14th ACM/IEEE Int Conf on Human-Robot Interaction (HRI). Piscataway, NJ: IEEE, 2019: 182−190
[6]	Zhao Xingyu, Robu V, Flynn D, et al. Assessing the safety and reliability of autonomous vehicles from road testing[C]//Proc of the 30th IEEE Int Symp on Software Reliability Engineering (ISSRE). Piscataway, NJ: IEEE, 2019: 13−23
[7]	Rong Guodong, Shin B H, Tabatabaee H, et al. LGSVL simulator: A high fidelity simulator for autonomous driving[C]//Proc of the 23rd IEEE Int Conf on Intelligent Transportation Systems (ITSC). Piscataway, NJ: IEEE, 2020: 1−6
[8]	Baltodano S, Sibi S, Martelaro N, et al. The RRADS platform: A real road autonomous driving simulator[C]//Proc of the 7th Int Conf on Automotive User Interfaces and Interactive Vehicular Applications. New York: ACM, 2015: 281−288
[9]	Son T D, Bhave A, Van der Auweraer H. Simulation-based testing framework for autonomous driving development[C]//Proc of the 2019 IEEE Int Conf on Mechatronics (ICM): Vol. 1. Piscataway, NJ: IEEE, 2019: 576−583
[10]	Dosovitskiy A, Ros G, Codevilla F, et al. CARLA: An open urban driving simulator[C]//Proc of the 1st Annual Conf on Robot Learning. New York: PMLR, 2017: 1−16
[11]	Hauer F, Schmidt T, Holzmüller B, et al. Did we test all scenarios for automated and autonomous driving systems?[C]//Proc of the 2019 IEEE Intelligent Transportation Systems Conf (ITSC). Piscataway, NJ: IEEE, 2019: 2950−2955
[12]	Tang Shuncheng, Zhang Zhenya, Zhang Yi, et al. A survey on automated driving system testing: Landscapes and trends[J]. arXiv preprint, arXiv: 2206.05961, 2022
[13]	Hauer F, Pretschner A, Holzmüller B. Re-using concrete test scenarios generally is a bad idea[C]//Proc of the 2020 IEEE Intelligent Vehicles Symp (IV). Piscataway, NJ: IEEE, 2020: 1305−1310
[14]	Lou Guannan, Deng Yao, Zheng Xi, et al. Testing of autonomous driving systems: Where are we and where should we go?[C]//Proc of the 30th ACM Joint European Software Engineering Conf and Symp on the Foundations of Software Engineering. New York: ACM, 2022: 31−43
[15]	Han Jiacheng, Zhou Zhiquan. Metamorphic fuzz testing of autonomous vehicles[C]//Proc of the 42nd IEEE/ACM Int Conf on Software Engineering Workshops. New York: ACM, 2020: 380−385
[16]	Li Guanpeng, Li Yiran, Jha S, et al. AV-FUZZER: Finding safety violations in autonomous driving systems[C]//Proc of the 31st IEEE Int Symp on Software Reliability Engineering (ISSRE). Piscataway, NJ: IEEE, 2020: 25−36
[17]	Hu Zhisheng, Guo Shengjian, Zhong Zhenyu, et al. Coverage-based scene fuzzing for virtual autonomous driving testing[J]. arXiv preprint, arXiv: 2106.00873, 2021
[18]	Zhong Ziyuan, Kaiser G, Ray B. Neural network guided evolutionary fuzzing for finding traffic violations of autonomous vehicles[J]. arXiv preprint, arXiv: 2109.06126, 2021
[19]	Kim S, Liu M, Rhee J, et al. DriveFuzz: Discovering autonomous driving bugs through driving quality-guided fuzzing[C]//Proc of the 2022 ACM SIGSAC Conf on Computer and Communications Security. New York: ACM, 2022: 1753−1767
[20]	Tian Haoxiang, Jiang Yan, Wu Guoquan, et al. MOSAT: Finding safety violations of autonomous driving systems using multi-objective genetic algorithm[C]//Proc of the 30th ACM Joint European Software Engineering Conf and Symp on the Foundations of Software Engineering. New York: ACM, 2022: 94−106
[21]	Zalewski M. American fuzzy lop[EB/OL]. [2023-02-22].https://lcamtuf.coredump.cx/afl/
[22]	LLVM. LibFuzzer – a library for coverage-guided fuzz testing [EB/OL]. [2023-02-27].https://llvm.org/docs/LibFuzzer.html
[23]	Society of Automotive Engineers (SAE). J3016 - Taxonomy and definitions for terms related to on-road motor vehicle automated driving systems[J]. SAE International, 2018, 4970(724): 1−5
[24]	LGSVL. Cyber RT Documents[EB/OL]. [2023-02-27].https://cyber-rt.readthedocs.io/en/latest/
[25]	ROS. Robot Operating System[EB/OL]. [2023-02-27].https://www.ros.org/
[26]	Cao Yulong, Wang Ningfei, Xiao Chaowei, et al. Invisible for both camera and lidar: Security of multi-sensor fusion based perception in autonomous driving under physical-world attacks[C]//Proc of the 2021 IEEE Symp on Security and Privacy (SP). Piscataway, NJ: IEEE, 2021: 176−194
[27]	Tian Yuchi, Pei Kexin, Jana S, et al. Deeptest: Automated testing of deep-neural-network-driven autonomous cars[C]//Proc of the 40th Int Conf on Software Engineering. New York: ACM, 2018: 303−314
[28]	Cao Yulong, Xiao Chaowei, Cyr B, et al. Adversarial sensor attack on lidar-based perception in autonomous driving[C]//Proc of the 2019 ACM SIGSAC Conf on Computer and Communications Security. New York: ACM, 2019: 2267−2281
[29]	Sun Jiachen, Cao Yulong, Alfred C, et al. Towards robust LiDAR-based perception in autonomous driving: General black-box adversarial sensor attack and countermeasures[C]//Proc of the 29th USENIX Security Symp (USENIX Security’ 20). Berkeley, CA: USENIX Association, 2020: 877−894
[30]	Zhao Yue, Zhu Hong, Liang Ruigang, et al. Seeing isn’t believing: Towards more robust adversarial attack against real world object detectors[C]//Proc of the 2019 ACM SIGSAC Conf on Computer and Communications Security. New York: ACM, 2019: 1989−2004
[31]	Zhang Yang, Foroosh H, Gong Boqing. CAMOU: Learning physical vehicle camouflages to adversarially attack detectors in the wild[C]//Proc of the 7th Int Conf on Learning Representations. Burlingame, CA: OpenReview, 2019 [2023-04-28].https://openreview.net/forum?id=SJgEl3A5tm
[32]	Zhong Ziyuan, Tang Yun, Zhou Yuan, et al. A Survey on Scenario-Based Testing for Automated Driving Systems in High-Fidelity Simulation[M]. arXiv preprint, arXiv: 2112.00964, 2021
[33]	Unity. Unity Real-Time Development Platform [EB/OL]. [2023-02-25].https://unity.com/
[34]	Unreal Engine. Unreal Engine [EB/OL]. [2023-02-26].https://www.unrealengine.com/en-US/
[35]	Queiroz R, Berger T, Czarnecki K. GeoScenario: An open DSL for autonomous driving scenario representation[C]//Proc of the 2019 IEEE Intelligent Vehicles Symp (IV). Piscataway, NJ: IEEE, 2019: 287−294
[36]	ASAM. OpenSCENARIO[EB/OL]. [2023-02-22].https://www.asam.net/standards/detail/openscenario
[37]	ASAM. OpenDRIVE[EB/OL]. [2023-02-28].https://www.asam.net/standards/detail/opendrive
[38]	OpenStreetMap. OpenStreetMap[EB/OL]. [2023-02-28].https://www.openstreetmap.org/
[39]	Bender P, Ziegler J, Stiller C. Lanelets: Efficient map representation for autonomous driving[C]//Proc of the 2014 IEEE Intelligent Vehicles Symp. Piscataway, NJ: IEEE, 2014: 420−425
[40]	Fioraldi A, D’Elia D C, Balzarotti D. The use of likely invariants as feedback for fuzzers[C]//Proc of the 30th USENIX Security Symp (USENIX Security’21). Berkeley, CA: USENIX Association, 2021: 2829−2846
[41]	Lee G, Shim W, Lee B. Constraint-guided directed greybox fuzzing[C]//Proc of the 30th USENIX Security Symp (USENIX Security’21). Berkeley, CA: USENIX Association, 2021: 3559−3576
[42]	Huang Heqing, Guo Yiyuan, Shi Qingkai, et al. BEACON: Directed grey-box fuzzing with provable path pruning[C]//Proc of the 2022 IEEE Symp on Security and Privacy (SP). Piscataway, NJ: IEEE, 2022: 36−50
[43]	Kukucka J, Pina L, Ammann P, et al. CONFETTI: Amplifying concolic guidance for fuzzers[C]//Proc of the 44th Int Conf on Software Engineering. New York: ACM, 2022: 438−450
[44]	Liang Jie, Wang Mingzhe, Zhou Chijin, et al. PATA: Fuzzing with path aware taint analysis[C]//Proc of the 2022 IEEE Symp on Security and Privacy (SP). Piscataway, NJ: IEEE, 2022: 1−17
[45]	Chen Ju, Wang Jinghan, Song Chengyu, et al. JIGSAW: Efficient and scalable path constraints fuzzing[C]//Proc of the 2022 IEEE Symp on Security and Privacy (SP). Piscataway, NJ: IEEE, 2022: 18−35
[46]	MacIver D R, Donaldson A F. Test-case reduction via test-case generation: Insights from the hypothesis reducer [C]//Proc of the 34th European Conf on Object-Oriented Programming (ECOOP 2020): Vol. 166. Dagstuhl, Germany: Schloss Dagstuhl–Leibniz-Zentrum für Informatik, 2020: 13: 1−13: 27
[47]	Jiang Zhiyuan, Jiang Xiyue, Hazimeh A, et al. Igor: Crash deduplication through root-cause clustering[C]//Proc of the 2021 ACM SIGSAC Conf on Computer and Communications Security. New York: ACM, 2021: 3318−3336
[48]	Serebryany K, Bruening D, Potapenko A, et al. AddressSanitizer: A fast address sanity checker[C]//Proc of the 2012 USENIX Annual Technical Conf (USENIX ATC’12). Berkeley, CA: USENIX Association, 2012: 309−318
[49]	Rebert A, Cha S K, Avgerinos T, et al. Optimizing seed selection for fuzzing[C]//Proc of the 23rd USENIX Security Symp (USENIX Security’14). Berkeley, CA: USENIX Association, 2014: 861−875
[50]	Zhu Xiaogang, Wen Sheng, Camtepe S, et al. Fuzzing: A survey for roadmap[J]. ACM Computing Surveys, 2022, 54(11s): 1−36
[51]	Caesar H, Bankiti V, Lang A H, et al. nuScenes: A multimodal dataset for autonomous driving[C]//Proc of the IEEE/CVF Conf on Computer Vision and Pattern Recognition. Piscataway, NJ: IEEE, 2020: 11621−11631
[52]	Huang Xinyu, Cheng Xinjing, Geng Qichuan, et al. The apolloscape dataset for autonomous driving[C]//Proc of the IEEE Conf on Computer Vision and Pattern Recognition Workshops. Piscataway, NJ: IEEE, 2018: 954−960
[53]	Muller R, Man Yanmao, Celik Z B, et al. Drivetruth: Automated autonomous driving dataset generation for security applications[C]//Proc of the Int Workshop on Automotive and Autonomous Vehicle Security (AutoSec). San Diego: NDSS, 2022: 1−7
[54]	Ouaknine A, Newson A, Rebut J, et al. Carrada dataset: Camera and automotive radar with range-angle-doppler annotations[C]//Proc of the 25th IEEE Int Conf on Pattern Recognition (ICPR). Piscataway, NJ: IEEE, 2021: 5068−5075
[55]	Sun Pei, Kretzschmar H, Dotiwalla X, et al. Scalability in perception for autonomous driving: Waymo open dataset[C]//Proc of the IEEE/CVF Conf on Computer Vision and Pattern Recognition. Piscataway, NJ: IEEE, 2020: 2446−2454
[56]	Wilson B, Qi W, Agarwal T, et al. Argoverse 2: Next generation datasets for self-driving perception and forecasting[J]. arXiv preprint, arXiv: 2301.00493, 2023
[57]	Xiao Pengchuan, Shao Zhenlei, Hao S, et al. Pandaset: Advanced sensor suite dataset for autonomous driving[C]//Proc of the 2021 IEEE 24th Int Conf on Intelligent Transportation Systems (ITSC). Piscataway, NJ: IEEE, 2021: 3095−3101
[58]	Yin Hang, Berger C. When to use what data set for your self-driving car algorithm: An overview of publicly available driving datasets[C]//Proc of the 2017 IEEE 20th Int Conf on Intelligent Transportation Systems (ITSC). Piscataway, NJ: IEEE, 2017: 1−8
[59]	Yu F, Chen Haofeng, Wang Xin, et al. Bdd100k: A diverse driving dataset for heterogeneous multitask learning[C]//Proc of the IEEE/CVF Conf on Computer Vision and Pattern Recognition. Piscataway, NJ: IEEE, 2020: 2636−2645
[60]	FHWA Operations. Traffic analysis tools: Next generation simulation[EB/OL]. [2023-03-07].https://ops.fhwa.dot.gov/trafficanalysistools/ngsim.htm
[61]	Bock J, Krajewski R, Moers T, et al. The inD dataset: A drone dataset of naturalistic road user trajectories at German intersections[C]//Proc of the 2020 IEEE Intelligent Vehicles Symp (IV). Piscataway, NJ: IEEE, 2020: 1929−1934
[62]	Maierhofer S, Klischat M, Althoff M. CommonRoad scenario designer: An open-source toolbox for map conversion and scenario creation for autonomous vehicles[C]//Proc of the 2021 IEEE Int Intelligent Transportation Systems Conf (ITSC). Piscataway, NJ: IEEE, 2021: 3176−3182
[63]	Böhme M, Pham V T, Roychoudhury A. Coverage-based greybox fuzzing as Markov chain[C]//Proc of the 2016 ACM SIGSAC Conf on Computer and Communications Security. New York: ACM, 2016: 1032−1043
[64]	NHTSA. ANSI D16 - 2017 Manual on Classification of Motor Vehicle Traffic Crashes[EB/OL]. [2023-03-01].https://www.nhtsa.gov/document/ansi-d16-2017-manual-classification-motor-vehicle-traffic-crashes
[65]	Amigó E, Gonzalo J, Artiles J, et al. A comparison of extrinsic clustering evaluation metrics based on formal constraints[J]. Information retrieval, 2009, 12: 461−486 doi: 10.1007/s10791-008-9066-8
[66]	Garcia J, Feng Yang, Shen Junjie, et al. A comprehensive study of autonomous vehicle bugs[C]//Proc of the ACM/IEEE 42nd Int Conf on Software Engineering. New York: ACM, 2020: 385−396
[67]	Cal ò A, Arcaini P, Ali S, et al. Generating avoidable collision scenarios for testingautonomous driving systems[C]//Proc of the 2020 IEEE Int Conf on Software Testing,Validation and Verification (ICST). Piscataway, NJ: IEEE, 2020: 375−386
[68]	Cal ò A, Arcaini P, Ali S, et al. Simultaneously searching and solving multiple avoidablecollisions for testing autonomous driving systems[C]//Proc of the 2020 Genetic andEvolutionary Computation Conf (GECCO). New York: ACM, 2020: 1055−1063

[1]	Wu Jingya, Lu Wenyan, Yan Guihai, Li Xiaowei. HyperTree: High Concurrent B+tree Index Accelerator[J]. Journal of Computer Research and Development, 2023, 60(7): 1661-1677. DOI: 10.7544/issn1000-1239.202111055
[2]	Yang Yongpeng, Jiang Dejun. A Method for Solving the wandering B+ tree Problem[J]. Journal of Computer Research and Development, 2023, 60(3): 539-554. DOI: 10.7544/issn1000-1239.202220555
[3]	Yan Wei, Zhang Xingjun, Ji Zeyu, Dong Xiaoshe, Ji Chenzhao. One-Direction Shift B⁺-Tree Based on Persistent Memory[J]. Journal of Computer Research and Development, 2021, 58(2): 371-383. DOI: 10.7544/issn1000-1239.2021.20200403
[4]	Te Rigen, Li Wei, and Li Xiongfei. Storage Model and Implementation of the Dynamic Ordered Tree[J]. Journal of Computer Research and Development, 2013, 50(5): 969-985.
[5]	Shen Yan, Song Shunlin, Zhu Yuquan. Mining Algorithm of Association Rules Based on Disk Table Resident FP-TREE[J]. Journal of Computer Research and Development, 2012, 49(6): 1313-1322.
[6]	Wang Hongqiang, Li Jianzhong, and Wang Hongzhi. Processing XPath over F&B-Index[J]. Journal of Computer Research and Development, 2010, 47(5): 866-877.
[7]	Zhou Da, Liang Zhichao, Meng Xiaofeng. HF-Tree: An Update-Efficient Index for Flash Memory[J]. Journal of Computer Research and Development, 2010, 47(5): 832-840.
[8]	Sun Xiaojuan, Sun Ninghui, Chen Mingyu. Optimization of B-NIDS for Multicore[J]. Journal of Computer Research and Development, 2007, 44(10): 1733-1740.
[9]	Ju Dapeng, Li Ming, Hu Jinfeng, Wang Dongsheng, Zheng Weimin, and Ma Yongquan. An Algorithm of B\++ Tree Management in P2P Environment[J]. Journal of Computer Research and Development, 2005, 42(8): 1438-1444.
[10]	Dong Daoguo, Liang Liuhong, and Xue Xiangyang. VAR-Tree—A New High-Dimensional Data Index Structure[J]. Journal of Computer Research and Development, 2005, 42(1): 10-17.

Supplements (1)

Supplements
Other Related Supplements
- Video
  https://www.bilibili.com/video/BV11u411a7eF/?spm_id_from=333.1387.search.video_card.click&vd_source=b27d2d557b6a91b712aa544f6c38158d

Cited By

Cited by

Periodical cited type(1)

1.	LUO Haoran，HU Shuisong，WANG Wenyong，TANG Yuke，ZHOU Junwei. Research on Multi-Core Processor Analysis for WCET Estimation. ZTE Communications. 2024(01): 87-94 .