Traffic Flow Prediction Based on Graph Prompt-Finetuning

Lai Peiyuan; Li Cheng; Wang Zenghui; Wang Changdong; Liao Dezhang

doi:10.7544/issn1000-1239.202440113

Journal of Computer Research and Development > 2024 > 61(8): 2020-2029. > DOI: 10.7544/issn1000-1239.202440113 CSTR: 32373.14.issn1000-1239.202440113

Lai Peiyuan, Li Cheng, Wang Zenghui, Wang Changdong, Liao Dezhang. Traffic Flow Prediction Based on Graph Prompt-Finetuning[J]. Journal of Computer Research and Development, 2024, 61(8): 2020-2029. DOI: 10.7544/issn1000-1239.202440113

Citation:

PDF (1760 KB)

Traffic Flow Prediction Based on Graph Prompt-Finetuning

1.
School of Information Engineering, Guangdong University of Technology, Guangzhou 510006
2.
Guangdong Provincial South China Technology Commercialization Center Co., Ltd., Guangzhou 511458
3.
School of Computer Science and Engineering, Sun Yat-sen University, Guangzhou 510006

Funds: This work was supported by the National Natural Science Foundation of China (62276277) and the Guangdong Provincial Engineering Research Center of Intelligent Matching for Technology Commercialization (2022A175).

More Information

Author Bio:
Lai Peiyuan: born in 1984. PhD. Senior engineer. Student member of CCF. His main research interests include data mining, recommendation algorithm, and graph neural network

Li Cheng: born in 1995. Master candidate. His main research interests include graph data mining and AI for medicine

Wang Zenghui: born in 1998. Bachelor, software engineer. His main research interests include data mining, recommendation algorithm, and graph clustering

Wang Changdong: born in 1984. PhD, associate professor, PhD supervisor. Distinguished member of CCF and IEEE senior member. His main research interests include graph data mining, AI for medicine, multi-view clustering, and AI for agriculture

Liao Dezhang: born in 1993. Bachelor, software engineer. His main research interests include data mining, recommendation algorithm, and graph clustering
Received Date: March 14, 2024
Revised Date: April 23, 2024
Available Online: May 16, 2024

Graphical Abstract

Abstract

Abstract

Traffic flow prediction is a crucial foundational aspect in the development of smart cities, holding significant implications for urban traffic management and user travel planning. The expanding dimensions of both time and space contribute to the increasing volume of traffic flow data, with real-time updates presenting a challenge in deploying cost-effective forecasting models to meet diverse traffic prediction demands. Inspired by the success of the emerging paradigm of graph-based finetuning in downstream tasks of natural language processing, we introduce, for the first time to our best knowledge, graph based finetuning to enhance the generalization capabilities of traditional traffic flow prediction models, called TPGPF (traffic flow prediction based on graph prompt-finetuning), which could enhance the generality and effectiveness of self-supervised learning. In the context of spatiotemporal multidimensional traffic flow prediction models, our approach involves pretraining the model based on historical datasets and introducing learnable prompt vectors. With the pretrained model solidified, a self-supervised learning model is guided by the introduced prompt vectors to adapt to new data prediction tasks, thereby enhancing the generality and effectiveness of traffic flow prediction models. Extensive experiments on five real-world public datasets validate the effectiveness of our work, which provides an effective method to overcome the cost challenge brought by quasi real-time training of traffic flow data.
- graph prompt,
- traffic flow prediction,
- finetuning,
- pre-trained model,
- self-supervised learning

FullText(HTML)

References (32)

References

[1]	Bernardo G, José C, Helena A. A survey on traffic flow prediction and classification[J], Intelligent Systems with Applications, 2023, 20: 200268
[2]	王萌,王昊奋,李博涵,等. 新一代知识图谱关键技术综述[J]. 计算机研究与发展,2022,59(9):1947−1965 doi: 10.7544/issn1000-1239.20210829 Wang Meng, Wang Haofen, Li Bohan, et al. Survey on key technologies of new generation knowledge graph[J]. Journal of Computer Research and Development, 2022, 59(9): 1947−1965 (in Chinese) doi: 10.7544/issn1000-1239.20210829
[3]	陈芮,王占全. Uni-LSDPM:基于预训练的统一在线学习会话退出预测模型[J]. 计算机研究与发展,2024,61(2):441−459 doi: 10.7544/issn1000-1239.202220834 Chen Rui, Wang Zhanquan. Uni-LSDPM: A unified online learning session dropout prediction model based on pre-training[J]. Journal of Computer Research and Development, 2024, 61(2): 441−459 (in Chinese) doi: 10.7544/issn1000-1239.202220834
[4]	Cheng Zeyang, Lu Jian, Zhou Huajian, et al. Short-term traffic flow prediction: An integrated method of econometrics and hybrid deep learning[J]. IEEE Transactions on Intelligent Transportation Systems, 2022, 23(6): 5231−5244 doi: 10.1109/TITS.2021.3052796
[5]	Cao Shuqin, Wu Libing, Wu Jia, et al. A spatio-temporal sequence-to-sequence network for traffic flow prediction[J], Information Sciences, 2022, 610: 185−203
[6]	Wang Hanqiu, Zhang Rongqing, Cheng Xiang, et al. Hierarchical traffic flow prediction based on spatial-temporal graph convolutional network[J]. IEEE Transactions on Intelligent Transportation Systems, 2022, 23(9): 16137−16147 doi: 10.1109/TITS.2022.3148105
[7]	汪鸣,彭舰,黄飞虎. 基于多时间尺度时空图网络的交通流量预测模型[J]. 计算机科学,2022,49(8):40−48 doi: 10.11896/jsjkx.220100188 Wang Ming, Peng Jian, Huang Feihu. Multi-time scale spatial-temporal graph neural network for traffic flow prediction[J]. Computer Science, 2022, 49(8): 40−48 (in Chinese) doi: 10.11896/jsjkx.220100188
[8]	赵文竹,袁冠,张艳梅,等. 多视角融合的时空动态图卷积网络城市交通流量预测[J]. 软件学报,2024,35(4):1751−1773 Zhao Wenzhu, Yuan Guan, Zhang Yanmei, et al. Multi-view fused spatial-temporal dynamic graph convolutional network for urban traffic flow prediction[J]. Journal of Software, 2024, 35(4): 1751−1773 (in Chinese)
[9]	Ji Jiahao, Wang Jingyuan, Huang Chao, et al. Spatio-temporal self-supervised learning for traffic flow prediction[C]//Proc of the AAAI Conf on Artificial Intelligence. Menlo Park, CA: AAAI, 2023, 37(4): 4356−4364
[10]	Lin Guancen, Lin Aijing, Gu Danlei, et al. Using support vector regression and K-nearest neighbors for short-term traffic flow prediction based on maximal information coefficient[J]. Information Sciences, 2022, 608: 517−531 doi: 10.1016/j.ins.2022.06.090
[11]	倪庆剑,彭文强,张志政,等. 基于信息增强传输的时空图神经网络交通流预测[J]. 计算机研究与发展,2022,59(2):282−293 Ni Qingjian, Peng Wenqiang, Zhang Zhizheng, et al. Spatial-temporal graph neural network for traffic flow prediction based on information enhanced transmission[J]. Journal of Computer Research and Development, 2022, 59(2): 282−293 (in Chinese)
[12]	Yan Biwei, WangGuijuan, Yu Jiguo, et al. Spatial-temporal chebyshev graph neural network for traffic flow prediction in IoT-based ITS[J]. IEEE Internet of Things Journal, 2022, 9(12): 9266−9279 doi: 10.1109/JIOT.2021.3105446
[13]	Zheng Haifeng, Lin Feng, Feng Xinxin, et al. A hybrid deep learning model with attention-based conv-LSTM networks for short-term traffic flow prediction[J]. IEEE Transactions on Intelligent Transportation Systems, 2021, 22(11): 6910−6920 doi: 10.1109/TITS.2020.2997352
[14]	杜圣东,李天瑞,杨燕,等. 一种基于序列到序列时空注意力学习的交通流预测模型[J]. 计算机研究与发展,2020,57(8):1715−1728 doi: 10.7544/issn1000-1239.2020.20200169 Du Shengdong, Li Tianrui, Yang Yan, et al. A sequence-to-sequence spatial-temporal attention learning model for urban traffic flow prediction[J]. Journal of Computer Research and Development, 2020, 57(8): 1715−1728 (in Chinese) doi: 10.7544/issn1000-1239.2020.20200169
[15]	Sun Mingchen, Zhou Kaixiong, He Xin, et al. GPPT: Graph pre-training and prompt tuning to generalize graph neural networks[C]//Proc of the 28th ACM SIGKDD Conf on Knowledge Discovery and Data Mining. New York: ACM, 2022: 1717−1727
[16]	Hu Ziniu, Dong Yuxiao, Wang Kuansan, et al. GPT-GNN: Generative pre-training of graph neural networks[C]//Proc of the 26th ACM SIGKDD Int Conf on Knowledge Discovery & Data Mining. New York: ACM, 2020: 1857−1867
[17]	Lu Yuanfu, Jiang Xunqiang, Fang Yuan, et al. Learning to pre-train graph neural networks[C]//Proc of the AAAI Conf on Artificial Intelligence. Menlo Park, CA: AAAI, 2021, 35(5): 4276−4284
[18]	Castro-Neto M, Jeong Y S, Jeong M K, et al. Online-SVR for short-term traffic flow prediction under typical and atypical traffic conditions[J]. Expert Systems with Applications, 2009, 36(3): 6164−6173 doi: 10.1016/j.eswa.2008.07.069
[19]	Kumar S V, Vanajakshi L. Short-term traffic flow prediction using seasonal ARIMA model with limited input data[J]. European Transport Research Review, 2015, 7(3): 1−9
[20]	Zhang Junbo, Zheng Yu, Qi Dekang. Deep spatio-temporal residual networks for citywide crowd flows prediction[C]//Proc of the AAAI Conf on Artificial Intelligence. Menlo Park, CA: AAAI, 2017, 31(1): 1655−1661
[21]	Yu Bing, Yin Haoteng, Zhu Zhanxing. Spatio-temporal graph convolutional networks: A deep learning framework for traffic forecasting [C]//Proc of Int Joint Conf on Artificial Intelligence Stockholm, Sweden: Ijcai.org, 2018: 3634−3640
[22]	Zheng Chuanpan, Fan Xiaoliang, Wang Cheng, et al. GMAN: A graph multi-attention network for traffic prediction[C]//Proc of the AAAI Conf on Artificial Intelligence. Menlo Park, CA: AAAI, 2020, 34(1): 1234−1241
[23]	Bai Lei, Yao Lina, Li Can, Wang Xianzhi, et al. Adaptive graph convolutional recurrent network for traffic forecasting[J]. Advances in Neural Information Processing Systems, 2020, 33: 17804−17815
[24]	Song Chao, Lin Youfang, Guo Shengnan, et al. Spatial-temporal synchronous graph convolutional networks: A new framework for spatial-temporal network data forecasting[C]//Proc of the AAAI Conf on Artificial Intelligence. Menlo Park, CA: AAAI, 2020, 34(1): 914−921
[25]	Sun Xiangguo, Zhang Jiawen, Wu Xixi, et al. Graph prompt learning: A comprehensive survey and beyond[J]. arXiv preprint, arXiv: 2311.16534, 2023
[26]	Liu Zemin, Yu Xingtong, Fang Yuan, et al. Graphprompt: Unifying pre-training and downstream tasks for graph neural networks[C]//Proc of the ACM Web Conf 2023. New York: ACM, 2023: 417−428
[27]	Chen Mouxiang, Liu Zemin, LiuChenghao, et al. ULTRA-DP: Unifying graph pre-training with multi-task graph dual prompt[J]. arXiv preprint, arXiv: 2310.14845, 2023
[28]	Chen Zihao, WangYing, Ma Fuyuan, Yuan Hao, et al. GPL-GNN: Graph prompt learning for graph neural network[J]. Knowledge-Based Systems, 2024, 286: 111391 doi: 10.1016/j.knosys.2024.111391
[29]	Fang Taoran, Zhang Yunchao, Yang Yang, et al. Universal prompt tuning for graph neural networks[J]. Advances in Neural Information Processing Systems, 2023, 36: 52464−52489
[30]	Yao Huaxiu, Tang Xianfeng, Wei Hua, et al. Revisiting spatial-temporal similarity: A deep learning framework for traffic prediction[C]//Proc of the AAAI Conf on Artificial Intelligence. Menlo Park, CA: AAAI, 2019: 5668−5675
[31]	Jiang Jiawei, Han Chengkai, Zhao W X, et al. PDFormer: Propagation delay-aware dynamic long-range transformer for traffic flow prediction[C]//Proc of the AAAI Conf on Artificial Intelligence. Menlo Park, CA: AAAI, 2023, 37(4): 4365−4373
[32]	Zhai Jianyang, Zheng Xiawu, Wang Changdong, et al. Knowledge prompt-tuning for sequential recommendation[C]// Proc of the 31st ACM Int Conf on Multimedia. New York: ACM, 2023, 6451−6461

[1]	Li Zhongnian, Huangfu Zhiyu, Yang Kaijie, Ying Peng, Sun Tongfeng, Xu Xinzheng. Semi-supervised Open Vocabulary Multi-label Learning Based on Graph Prompting[J]. Journal of Computer Research and Development, 2025, 62(2): 432-442. DOI: 10.7544/issn1000-1239.202440123
[2]	Zhang Xiaohui, Yi Jiangyan, Tao Jianhua, Zhou Junzuo. Elastic Orthogonal Weight Modification Continual Learning Algorithm in the Context of Synthetic Speech Detection[J]. Journal of Computer Research and Development, 2025, 62(2): 336-345. DOI: 10.7544/issn1000-1239.202330311
[3]	Cheng Yudong, Zhou Fang. Semi-Supervised Learning-Based Method for Unknown Anomaly Detection[J]. Journal of Computer Research and Development, 2024, 61(7): 1670-1680. DOI: 10.7544/issn1000-1239.202330627
[4]	Chen Rui, Wang Zhanquan. Uni-LSDPM: A Unified Online Learning Session Dropout Prediction Model Based on Pre-Training[J]. Journal of Computer Research and Development, 2024, 61(2): 441-459. DOI: 10.7544/issn1000-1239.202220834
[5]	Zhu Minghang, Liu Xin, Yu Zhenning, Xu Xing, Zheng Shukai. Cross Face-Voice Matching Method via Bi-Pseudo Label Based Self-Supervised Learning[J]. Journal of Computer Research and Development, 2023, 60(11): 2638-2649. DOI: 10.7544/issn1000-1239.202220411
[6]	Jiao Pengfei, Liu Huan, Lü Le, Gao Mengzhou, Zhang Jilin, Liu Dong. Globally Enhanced Heterogeneous Temporal Graph Neural Networks Based on Contrastive Learning[J]. Journal of Computer Research and Development, 2023, 60(8): 1808-1821. DOI: 10.7544/issn1000-1239.202330226
[7]	Du Shengdong, Li Tianrui, Yang Yan, Wang Hao, Xie Peng, Horng Shi-Jinn. A Sequence-to-Sequence Spatial-Temporal Attention Learning Model for Urban Traffic Flow Prediction[J]. Journal of Computer Research and Development, 2020, 57(8): 1715-1728. DOI: 10.7544/issn1000-1239.2020.20200169
[8]	Chen Liwei, Feng Yansong, and Zhao Dongyan. Extracting Relations from the Web via Weakly Supervised Learning[J]. Journal of Computer Research and Development, 2013, 50(9): 1825-1835.
[9]	Wang Lihong, Zhao Xianjia, Wu Shuanhu. Semi-supervised Learning of Promoter Sequences Based on EM Algorithm[J]. Journal of Computer Research and Development, 2009, 46(11): 1942-1948.
[10]	Li Ming and Zhou Zhihua. Online Semi-Supervised Learning with Multi-Kernel Ensemble[J]. Journal of Computer Research and Development, 2008, 45(12): 2060-2068.