GraphMLP-Mixer: A Graph-MLP Architecture for Efficient Multi-Behavior Sequential Recommendation Method

Lu Xiaokai; Feng Jun; Han Yongqiang; Wang Hao; Chen Enhong

doi:10.7544/issn1000-1239.202440137

Journal of Computer Research and Development > 2024 > 61(8): 1917-1929. > DOI: 10.7544/issn1000-1239.202440137

Lu Xiaokai, Feng Jun, Han Yongqiang, Wang Hao, Chen Enhong. GraphMLP-Mixer: A Graph-MLP Architecture for Efficient Multi-Behavior Sequential Recommendation Method[J]. Journal of Computer Research and Development, 2024, 61(8): 1917-1929. DOI: 10.7544/issn1000-1239.202440137

Citation:

PDF (1078 KB)

GraphMLP-Mixer: A Graph-MLP Architecture for Efficient Multi-Behavior Sequential Recommendation Method

Lu Xiaokai^{1, 2,},
Feng Jun^{1, 2,},
Han Yongqiang^1,,
Wang Hao^1, ,,
Chen Enhong^1,

1.
State Key Laboratory of Cognitive Intelligence (University of Science and Technology of China), Hefei 230000
2.
YouZhiCai Science and Technology Development Co., Ltd, Hefei 230000

Funds: This work was supported by the National Natural Science Foundation of China (62202443, U23A20319), the Anhui Provincial Science and Technology Major Project (2023z020006), and the CCF-Tencent Rhino-Bird Open Research Fund (RAGR20230124).

More Information

Author Bio:
Lu Xiaokai: born in 1980. PhD candidate, senior engineer. His main research interests include intelligent recommendation, data mining, knowledge graph, large model of industrial products, and intelligent supply chain

Feng Jun: born in 1983. Master candidate, senior engineer. His main research interests include data mining, machine learning, and digital supply chain intelligence. (fengjun@youzhicai.com)

Han Yongqiang: born in 1999. Master. Member of CCF. His main research interests include information retrieval and recommendation algorithms. （harley@mail.ustc.edu.cn）

Wang Hao: born in 1993. PhD, special associate researcher. His main research interests include representation learning, graph mining, and large language models. （wanghao3@ustc.edu.cn）

Chen Enhong: born in 1968. PhD, professor. His main research interests include machine learning and knowledge discovery. （cheneh@ustc.edu.cn）
Received Date: March 14, 2024
Revised Date: May 22, 2024
Available Online: July 04, 2024

Graphical Abstract

Abstract

Abstract

In the domain of multi-behavior sequence recommendation, Graph Neural Networks (GNNs) have been widespreadly adopted, yet they have limitations, notably in terms of adequately modeling the collaborative signals that exist between different sequences and addressing the challenges posed by long-distance dependencies. To bridge these gaps, a novel framework named GraphMLP-Mixer has been introduced. This innovative framework begins by constructing a global item graph, which is designed to bolster the model’s capacity to encapsulate the collaborative signals that are present across sequences. It then merges the perceptron-mixer architecture with graph neural networks, resulting in a graph-perceptron mixer model capable of delving deep into the intricacies of user interests. GraphMLP-Mixer stands out for its two principal strengths: It not only succeeds in effectively capturing the global dependencies inherent in user behaviors but also manages to alleviate the issue of excessive information compression. Furthermore, the framework boasts remarkable improvements in terms of time and space efficiency, with its complexity scaling in a linear fashion with the number of user interactions, thus outperforming existing GNN-based models in the realm of multi-behavior sequence recommendation. The robustness and efficiency of GraphMLP-Mixer in tackling the complexities of multi-behavior sequence recommendation have been thoroughly validated through extensive experimentation on three diverse and publicly available datasets.
- multi-behavior modeling,
- sequential recommendation,
- graph neural network,
- MLP architecture,
- global item graph

FullText(HTML)

References (37)

References

[1]	Wang Shoujin, Hu Liang, Wang Yan, et al. Sequential recommender systems: Challenges, progress and prospects[J]. arXiv preprint, arXiv: 2001.04830, 2019
[2]	Hidasi B, Karatzoglou A, Baltrunas L, et al. Session-based recommendations with recurrent neural networks[J]. arXiv preprint, arXiv: 1511.06939, 2015
[3]	Tang Jiaxi, Wang Ke. Personalized top-n sequential recommendation via convolutional sequence embedding[C]//Proc of the 11th ACM Int Conf on Web Search and Data Mining. New York: ACM, 2018: 565−573
[4]	Vaswani A, Shazeer N, Parmar N, et al. Attention is all you need[J]. Advances in Neural Information Processing Systems, 2017, 30
[5]	Kang W, McAuley J. Self-attentive sequential recommendation[C]//Proc of 2018 IEEE Int Conf on Data Mining (ICDM). Piscataway, NJ: IEEE, 2018: 197−206
[6]	Sun Fei, Liu Jun, Wu Jian, et al. BERT4Rec: Sequential recommendation with bidirectional encoder representations from transformer[C]//Proc of the 28th ACM Int Conf on Information and Knowledge Management. New York: ACM, 2019: 1441−1450
[7]	Wu Zonghan, Pan Shirui, Chen Fengwen, et al. A comprehensive survey on graph neural networks[J]. IEEE Transactions on Neural Networks and Learning Systems, 2020, 32(1): 4−24
[8]	Wu Shu, Tang Yuyuan, Zhu Yanqiao, et al. Session-based recommendation with graph neural networks[C]//Proc of the AAAI Conf on Artificial Intelligence. Palo Alto, CA: AAAI, 2019, 33(1): 346−353
[9]	Xu Chengfeng, Zhao Pengpeng, Liu Yanchi, et al. Graph contextualized self-attention network for session-based recommendation[C]//Freiburg: IJCAI. 2019, 19: 3940−3946
[10]	Zhao Zhe, Cheng Zhiyuan, Hong Lichan, et al. Improving user topic interest profiles by behavior factorization[C]//Proc of the 24th Int Conf on World Wide Web. New York: ACM, 2015: 1406−1416
[11]	Jin Bowen, Gao Chen, He Xiangnan, et al. Multi-behavior recommendation with graph convolutional networks[C]//Proc of the 43rd Int ACM SIGIR Conf on Research and Development in Information Retrieval. New York: ACM, 2020: 659−668
[12]	Xia Lianghao, Huang Chao, Xu Yong, et al. Knowledge-enhanced hierarchical graph transformer network for multi-behavior recommendation[C]//Proc of the AAAI Conf on Artificial Intelligence. Palo Alto, CA: AAAI, 2021, 35(5): 4486−4493
[13]	Wei Wei, Huang Chao, Xia Lianghao, et al. Contrastive meta learning with behavior multiplicity for recommendation[C]//Proc of the 15th ACM Int Conf on web search and data mining. New York: ACM, 2022: 1120−1128
[14]	Xuan Hongrui, Liu Yi, Li Bohan, et al. Knowledge enhancement for contrastive multi-behavior recommendation[C]//Proc of the 16th ACM Int Conf on Web Search and Data Mining. New York: ACM, 2023: 195−203
[15]	Xu Keyulu, Hu Weihua, Leskovec J, et al. How powerful are graph neural networks?[J]. arXiv preprint, arXiv: 1810.00826, 2018
[16]	Tolstikhin I O, Houlsby N, Kolesnikov A, et al. MLP-Mixer: An all-MLP architecture for vision[J]. Advances in Neural Information Processing Systems, 2021, 34: 24261−24272
[17]	Rendle S, Freudenthaler C, Schmidt-Thieme L. Factorizing personalized Markov chains for next-basket recommendation[C]//Proc of the 19th Int Conf on World Wide Web. New York: ACM, 2010: 811−820
[18]	He Ruining, McAuley J. Fusing similarity models with Markov chains for sparse sequential recommendation[C]//Proc of 2016 IEEE 16th Int Conf on Data Mining (ICDM). Piscataway, NJ: IEEE, 2016: 191−200
[19]	Huang Chao, Chen Jiahui, Xia Lianghao, et al. Graph-enhanced multi-task learning of multi-level transition dynamics for session-based recommendation[C]//Proc of the AAAI Conf on Artificial Intelligence. Palo Alto, CA: AAAI, 2021, 35(5): 4123−4130
[20]	Wang Ziyang, Wei Wei, Cong Gao, et al. Global context enhanced graph neural networks for session-based recommendation[C]//Proc of the 43rd Int ACM SIGIR Conf on Research and Development in Information Retrieval. New York: ACM, 2020: 169−178
[21]	Gao Chen, He Xiangnan, Gan Dahua, et al. Neural multi-task recommendation from multi-behavior data[C]//Proc of 2019 IEEE 35th Int Conf on Data Engineering (ICDE). Piscataway, NJ: IEEE, 2019: 1554−1557
[22]	Guo Long, Hua Lifeng, Jia Rongfei, et al. Buying or browsing?: Predicting real-time purchasing intent using attention-based deep network with multiple behavior[C]//Proc of the 25th ACM SIGKDD Int Conf on Knowledge Discovery & Data Mining. New York: ACM, 2019: 1984−1992
[23]	Xia Lianghao, Huang Chao, Xu Yong, et al. Multiplex behavioral relation learning for recommendation via memory augmented transformer network[C]//Proc of the 43rd Int ACM SIGIR Conf on Research and Development in Information Retrieval. New York: ACM, 2020: 2397−2406
[24]	Karypis G, Kumar V. A fast and high quality multilevel scheme for partitioning irregular graphs[J]. SIAM Journal on Scientific Computing, 1998, 20(1): 359−392 doi: 10.1137/S1064827595287997
[25]	Kipf T N, Welling M. Semi-supervised classification with graph convolutional networks[J]. arXiv preprint, arXiv: 1609.02907, 2016
[26]	Bresson X, Laurent T. Residual gated graph convnets[J]. arXiv preprint, arXiv: 1711.07553, 2017
[27]	Hu Weihua, Liu Bowen, Gomes J, et al. Strategies for pre-training graph neural networks[J]. arXiv preprint, arXiv: 1905.12265, 2019
[28]	Dwivedi V P, Bresson X. A generalization of transformer networks to graphs[J]. arXiv preprint, arXiv: 2012.09699, 2020
[29]	Rampášek L, Galkin M, Dwivedi V P, et al. Recipe for a general, powerful, scalable graph transformer[J]. Advances in Neural Information Processing Systems, 2022, 35: 14501−14515
[30]	Dwivedi V P, Luu A T, Laurent T, et al. Graph neural networks with learnable structural and positional representations[J]. arXiv preprint, arXiv: 2110.07875, 2021
[31]	Dwivedi V P, Joshi C K, Luu A T, et al. Benchmarking graph neural networks[J]. Journal of Machine Learning Research, 2023, 24(43): 1−48
[32]	Hendrycks D, Gimpel K. Gaussian error linear units (GELUS)[J]. arXiv preprint, arXiv: 1606.08415, 2016
[33]	Ba J L, Kiros J R, Hinton G E. Layer normalization[J]. arXiv preprint, arXiv: 1607.06450, 2016
[34]	Liu Hanxiao, Dai Zihang, So D, et al. Pay attention to MLPS[J]. Advances in Neural Information Processing Systems, 2021, 34: 9204−9215
[35]	Xie Xu, Sun Fei, Liu Zhaoyang, et al. Contrastive learning for sequential recommendation[C]//Proc of 2022 IEEE 38th Int Conf on Data Engineering (ICDE). Piscataway, NJ: IEEE, 2022: 1259−1273
[36]	Zhou Kun, Yu Hui, Zhao Xin, et al. Filter-enhanced MLP is all you need for sequential recommendation[C]//Proc of the ACM Web Conf 2022. New York: ACM, 2022: 2388−2399
[37]	Du Xinyu, Yuan Huanhuan, Zhao Pengpeng, et al. Frequency enhanced hybrid attention network for sequential recommendation[C]//Proc of the 46th Int ACM SIGIR Conf on Research and Development in Information Retrieval. New York: ACM, 2023: 78−88

[1]	Wu Tianxing, Cao Xudong, Bi Sheng, Chen Ya, Cai Pingqiang, Sha Hangyu, Qi Guilin, Wang Haofen. Constructing Health Management Information System for Major Chronic Diseases Based on Large Language Model[J]. Journal of Computer Research and Development. DOI: 10.7544/issn1000-1239.202440570
[2]	Zhao Yun, Liu Dexi, Wan Changxuan, Liu Xiping, Liao Guoqiong. Mental Health Text Matching Model Integrating Characters’ Mental Portrait[J]. Journal of Computer Research and Development, 2024, 61(7): 1812-1824. DOI: 10.7544/issn1000-1239.202220987
[3]	Fu Tao, Chen Zhaojiong, Ye Dongyi. GAN-Based Bidirectional Decoding Feature Fusion Extrapolation Algorithm of Chinese Landscape Painting[J]. Journal of Computer Research and Development, 2022, 59(12): 2816-2830. DOI: 10.7544/issn1000-1239.20210830
[4]	Gan Xinbiao, Tan Wen, Liu Jie. Bidirectional-Bitmap Based CSR for Reducing Large-Scale Graph Space[J]. Journal of Computer Research and Development, 2021, 58(3): 458-466. DOI: 10.7544/issn1000-1239.2021.20200090
[5]	Zhou Donghao, Han Wenbao, Wang Yongjun. A Fine-Grained Information Diffusion Model Based on Node Attributes and Content Features[J]. Journal of Computer Research and Development, 2015, 52(1): 156-166. DOI: 10.7544/issn1000-1239.2015.20130915
[6]	Li Yaxiong, Zhang Jianqiang, Pan Deng, Hu Dan. A Study of Speech Recognition Based on RNN-RBM Language Model[J]. Journal of Computer Research and Development, 2014, 51(9): 1936-1944. DOI: 10.7544/issn1000-1239.2014.20140211
[7]	Huang He, Sun Yu'e, Chen Zhili, Xu Hongli, Xing Kai, Chen Guoliang. Completely-Competitive-Equilibrium-Based Double Spectrum Auction Mechanism[J]. Journal of Computer Research and Development, 2014, 51(3): 479-490.
[8]	Zhu Feng, Luo Limin, Song Yuqing, Chen Jianmei, Zuo Xin. Adaptive Spatially Neighborhood Information Gaussian Mixture Model for Image Segmentation[J]. Journal of Computer Research and Development, 2011, 48(11): 2000-2007.
[9]	Ma Xiao, Wang Xuan, and Wang Xiaolong. The Information Model for a Class of Imperfect Information Game[J]. Journal of Computer Research and Development, 2010, 47(12).
[10]	Ma Liang, Chen Qunxiu, and Cai Lianhong. An Improved Model for Adaptive Text Information Filtering[J]. Journal of Computer Research and Development, 2005, 42(1): 79-84.