An Intelligent Traffic Signal Coordination Method Based on Asynchronous Decision-Making

Gao Han; Luo Juan; Cai Qianya; Zheng Yanliu

doi:10.7544/issn1000-1239.202220773

Journal of Computer Research and Development > 2023 > 60(12): 2797-2805. > DOI: 10.7544/issn1000-1239.202220773

Gao Han, Luo Juan, Cai Qianya, Zheng Yanliu. An Intelligent Traffic Signal Coordination Method Based on Asynchronous Decision-Making[J]. Journal of Computer Research and Development, 2023, 60(12): 2797-2805. DOI: 10.7544/issn1000-1239.202220773

Citation:

PDF (2362 KB)

An Intelligent Traffic Signal Coordination Method Based on Asynchronous Decision-Making

College of Computer Science and Electronic Engineering, Hunan University, Changsha 410082

Funds: This work was supported by the National Natural Science Foundation of China (61972140).

More Information

Author Bio:
Gao Han: born in 1998. Master. Her main research interest includes traffic control method in Internet of vehicles environment

Luo Juan: born in 1974. PhD, professor, PhD supervisor. Senior member of CCF. Her main research interests include Internet of things, edge computing, satellite Internet, and AI

Cai Qianya: born in 1997. Master. Her main research interest includes Internet of vehicles

Zheng Yanliu: born in 1995. PhD candidate. Her main research interests include Internet of vehicles and spatio-temporal data mining
Received Date: August 31, 2022
Revised Date: February 05, 2023
Available Online: September 19, 2023

Graphical Abstract

Abstract

Abstract

The intelligent traffic signal control system is a component of the intelligent traffic system (ITS), offering real-time services for the creation of a safe and efficient traffic environment. However, due to restricted communication, conventional adaptive traffic signal-controlled methods are unable to fulfill the complex and changing traffic requirements. A multi-agent adaptive coordination method (ADM) based on asynchronous decision-making and edge computing is presented to address the issues of communication delay and a decrease in signal utilization. Firstly, the end-side-cloud architecture is proposed for real-time environmental information collection and related processing. Then, to enhance the agent coordination process, asynchronous communication is implemented. An approach for calculating the decision cycle of the agent is presented, and an asynchronous decision mechanism employing multiple agents’ decision cycles is devised. The experimental results show that edge computing technology provides a good solution for traffic signal control scenarios with high real-time requirements. In addition, compared with the fixed time (FT) and independent Q-learning decision algorithm (IQA), ADM achieves collaboration among the agents based on the asynchronous decision mechanism and the neighbor information base, and reduces the average vehicle waiting length and improves intersection time utilization.
- asynchronous decision-making,
- coordination control,
- edge computing,
- reinforcement learning,
- traffic signal control

FullText(HTML)

References (24)

References

[1]	Cao Zhiguang, Jiang Siwei, Zhang Jie, et al. A unified framework for vehicle rerouting and traffic light control to reduce traffic congestion[J]. IEEE Transactions on Intelligent Transportation Systems, 2017, 18(7): 1958−1973 doi: 10.1109/TITS.2016.2613997
[2]	郭宪. 深入浅出强化学习: 原理入门[M]. 北京: 电子工业出版社, 2018 Guo Xian. Intensive Learning in Simple Terms: Introduction to Principles [M]. Beijing: Electronic Industry Press, 2018 (in Chinese)
[3]	Genders W, Razavi S. Using a deep reinforcement learning agent for traffic signal control[J]. arXiv preprint, arXiv:, 1611, 01142: Article No.2016
[4]	Zhou Pengyuan, Chen Xianfu, Liu Zhi, et al. DRLE: Decentralized reinforcement learning at the edge for traffic light control in the IoV[J]. IEEE Transactions on Intelligent Transportation Systems, 2021, 22(4): 2262−2273 doi: 10.1109/TITS.2020.3035841
[5]	Jaleel A, Hassan M A, Mahmood T, et al. Reducing congestion in an intelligent traffic system with collaborative and adaptive signaling on the edge[J]. IEEE Access, 2020, 8: 205396−205410 doi: 10.1109/ACCESS.2020.3037348
[6]	Tan Tian, Bao Feng, Deng Yue, et al. Cooperative deep reinforcement learning for large-scale traffic grid signal control[J]. IEEE Transactions on Cybernetics, 2020, 50(6): 2687−2700 doi: 10.1109/TCYB.2019.2904742
[7]	王莹多. 基于深度强化学习的路口自适应控制[D]. 大连: 大连理工大学, 2017 Wang Yingduo. Adaptive control of intersections based on deep reinforcement learning [D]. Dalian: Dalian University of Technology, 2017 (in Chinese)
[8]	喻金忠. 基于多智能体的城市路网交通信号控制研究[D]. 南京: 东南大学, 2019 Yu Jinzhong. Research on traffic signal control of urban road network based on multi-agent [D]. Nanjing: Southeast University, 2019 (in Chinese)
[9]	夏新海. 城市交通信号局部博弈交互下的学习协调控制[J]. 计算机工程与应用,2020,56(23):245−252 doi: 10.3778/j.issn.1002-8331.2001-0061 Xia Xinhai. Learning coordinated control under local game interaction of urban traffic signals[J]. Computer Engineering and Applications, 2020, 56(23): 245−252 (in Chinese) doi: 10.3778/j.issn.1002-8331.2001-0061
[10]	Qu Zhaowei, Pan Zhaotian, Chen Yongheng, et al. A distributed control method for urban networks using multi-agent reinforcement learning based on regional mixed strategy nash-equilibrium[J]. IEEE Access, 2020, 8: 19750−19766 doi: 10.1109/ACCESS.2020.2968937
[11]	卞宇. 基于博弈论的区域交通信号协调及优化控制研究[D]. 南京: 南京邮电大学, 2020 Bian Yu. Research on regional traffic signal coordination and optimal control based on game theory [D]. Nanjing: Nanjing University of Posts and Telecommunications, 2020 (in Chinese)
[12]	Chu Tianshu, Wang Jie, Codeca L, et al. Multi-agent deep reinforcement learning for large-scale traffic signal control [J], IEEE Transactions on Intelligent Transportation Systems, 2020, 21(3): 1086–1095
[13]	Wu Tong, Zhou Pan, Liu Kai, et al. Multi-agent deep reinforcement learning for urban traffic light control in vehicular networks[J]. IEEE Transactions on Vehicular Technology, 2020, 69(8): 8243−8256 doi: 10.1109/TVT.2020.2997896
[14]	Zheng Guanjie , Xiong Yuanhao , Zang Xinshi, et al. Learning phase competition for traffic signal control [C] //Proc of the 28th ACM Int Conf on Information and Knowledge Management. New York: ACM, 2019: 1963–1972
[15]	Genders W, Razavi S. Policy analysis of adaptive traffic signal control using reinforcement learning[J]. Journal of Computing in Civil Engineering, 2020, 34(1): 04019046 doi: 10.1061/(ASCE)CP.1943-5487.0000859
[16]	Wang Min, Wu Libing, Li Jianxin, et al. Traffic signal control with reinforcement learning based on region-aware cooperative strategy[J]. IEEE Transactions on Intelligent Transportation Systems, 2022, 23(7): 6774−6785 doi: 10.1109/TITS.2021.3062072
[17]	Wei Hua, Xu Nan, Zhang Huichu, et al. CoLight: Learning network-level cooperation for traffic signal control [C] //Proc of the 28th ACM Int Conf on Information and Knowledge Management. New York: ACM, 2019: 1913–1922
[18]	Devailly F, Larocque D, Charlin l. IG-RL: Inductive graph reinforcement learning for massive-scale traffic signal control[J]. IEEE Transactions on Intelligent Transportation Systems, 2021, 23(7): 7496−7507
[19]	Zeng Zheng. GraphLight: Graph-based reinforcement learning for traffic signal control [C] //Proc of the 6th Int Conf on Computer and Communication Systems. New York: ACM, 2021: 645–650
[20]	Nishi T, Otaki K, Hayakawa K, et al. Traffic signal control based on reinforcement learning with graph convolutional neural nets [C] //Proc of the 21st Int Conf Intelligent Transportation Systems. New York: ACM, 2018: 877–883
[21]	Mnih V, Badia A P, Mirza M, et al. Asynchronous methods for deep reinforcement learning [C] //Proc of the 33rd Int Conf on Machine Learning. New York: ACM, 2016: 1928–1937
[22]	Zhu Jichen, Ma Chengyuan, Yang Xiaoguang, et al. An asynchronous cooperative signal control framework in urban road network [C] // Proc of the 6th Int Conf on Transportation Information and Safety. Piscataway, NJ: IEEE, 2021: 1105–1111
[23]	Genders W, Razavi S. Asynchronous n-step Q-learning adaptive traffic signal control[J]. Journal of Intelligent Transportation Systems, 2019, 23(4): 319−331 doi: 10.1080/15472450.2018.1491003
[24]	Wang Tong, Cao Jiahua, Hussain A. Adaptive traffic signal control for large-scale scenario with cooperative group-based multi-agent reinforcement learning[J]. Transportation Research Part C Emerging Technologies, 2021, 125(3): 103046

[1]	Lin Liansheng, Zheng Huanqin, Su Shen, Lei Kai, Chen Xiaofeng, Tian Zhihong. An On-Chain Mechanism Against DeFi Price Manipulation Attacks[J]. Journal of Computer Research and Development, 2025, 62(2): 443-457. DOI: 10.7544/issn1000-1239.202330291
[2]	Song Shuwei, Ni Xiaoze, Chen Ting. Gas Optimization for Smart Contracts: A Survey[J]. Journal of Computer Research and Development, 2023, 60(2): 311-325. DOI: 10.7544/issn1000-1239.202220887
[3]	Ying Chenhao, Xia Fuyuan, Li Jie, Si Xueming, Luo Yuan. Incentive Mechanism Based on Truth Estimation of Private Data for Blockchain-Based Mobile Crowdsensing[J]. Journal of Computer Research and Development, 2022, 59(10): 2212-2232. DOI: 10.7544/issn1000-1239.20220493
[4]	Feng Jingyu, Yang Jinwen, Zhang Ruitong, Zhang Wenbo. A Spectrum Sharing Incentive Scheme Against Location Privacy Leakage in IoT Networks[J]. Journal of Computer Research and Development, 2020, 57(10): 2209-2220. DOI: 10.7544/issn1000-1239.2020.20200453
[5]	Hai Mo, Zhu Jianming. A Propagation Mechanism Combining an Optimal Propagation Path and Incentive in Blockchain Networks[J]. Journal of Computer Research and Development, 2019, 56(6): 1205-1218. DOI: 10.7544/issn1000-1239.2019.20180419
[6]	He Yunhua, Li Mengru, Li Hong, Sun Limin, Xiao Ke, Yang Chao. A Blockchain Based Incentive Mechanism for Crowdsensing Applications[J]. Journal of Computer Research and Development, 2019, 56(3): 544-554. DOI: 10.7544/issn1000-1239.2019.20170670
[7]	He Haiwu, Yan An, Chen Zehua. Survey of Smart Contract Technology and Application Based on Blockchain[J]. Journal of Computer Research and Development, 2018, 55(11): 2452-2466. DOI: 10.7544/issn1000-1239.2018.20170658
[8]	Xiong Jinbo, Ma Rong, Niu Ben, Guo Yunchuan, Lin Li. Privacy Protection Incentive Mechanism Based on User-Union Matching in Mobile Crowdsensing[J]. Journal of Computer Research and Development, 2018, 55(7): 1359-1370. DOI: 10.7544/issn1000-1239.2018.20180080
[9]	Wang Bo, Huang Chuanhe, Yang Wenzhong, Dan Feng, and Xu Liya. An Incentive-Cooperative Forwarding Model Based on Punishment Mechanism in Wireless Ad Hoc Networks[J]. Journal of Computer Research and Development, 2011, 48(3): 398-406.
[10]	Yue Guangxue, Li Renfa, Chen Zhi, Zhou Xu. Analysis of Free-riding Behaviors and Modeling Restrain Mechanisms for Peer-to-Peer Networks[J]. Journal of Computer Research and Development, 2011, 48(3): 382-397.

Cited By

Cited by

Periodical cited type(2)

1.	李硕，王馨爽. 多场景融合的码号数据分发架构及关键技术研究. 数据通信. 2024(06): 1-3+11 .
2.	俞惠芳，李磊. 基于椭圆曲线签密的跨链医疗数据共享方案. 通信学报. 2024(12): 57-66 .