High-Order Tensor Analysis Method for Information System Recommendations and Decisions

Wang Beilun; Zhang Jiaqi; Cai Yinghao; Wang Zhaoyang; Tan Xiao; Shen Dian

doi:10.7544/issn1000-1239.202330624

Journal of Computer Research and Development > 2024 > 61(7): 1697-1712. > DOI: 10.7544/issn1000-1239.202330624 CSTR: 32373.14.issn1000-1239.202330624

Wang Beilun, Zhang Jiaqi, Cai Yinghao, Wang Zhaoyang, Tan Xiao, Shen Dian. High-Order Tensor Analysis Method for Information System Recommendations and Decisions[J]. Journal of Computer Research and Development, 2024, 61(7): 1697-1712. DOI: 10.7544/issn1000-1239.202330624

Citation:

PDF (2388 KB)

High-Order Tensor Analysis Method for Information System Recommendations and Decisions

School of Computer Science and Engineering, Southeast University, Nanjing 210096

Funds: This work was supported by the National Natural Science Foundation of China (61906040, 61972085, 62276063, 6227072991), the Natural Science Foundation of Jiangsu Province (BK20190345, BK20190335, BK20221457), the National Key Research and Development Program of China (2022YFF0712400), and the Fundamental Research Funds for the Central Universities (2242021R41177).

More Information

Author Bio:
Wang Beilun: born in 1990. PhD, associate professor. His main research interests include large-scale machine learning, graphical model, and multi-task learning

Zhang Jiaqi: born in 1997. PhD candidate. His main research interests include graphical model and large-scale optimization

Cai Yinghao: born in 2002. Undergraduate. His main research interests include graph neural network, knowledge graph, and machine learning

Wang Zhaoyang: born in 2000. Master candidate. His main research interests include machine learning and software-hardware algorithm acceleration

Tan Xiao: born in 2000. PhD candidate. Her main research interests include graphical model, large-scale machine learning, and meta-learning

Shen Dian: born in 1988. PhD, associate professor. His main research interests include cloud (edge) computing, network intelligence, and intelligent algorithms and applications
Received Date: July 30, 2023
Revised Date: February 01, 2024
Available Online: May 16, 2024

Graphical Abstract

Abstract

Abstract

Tensor data (or multi-dimensional array data) are often generated in information systems of various industries, such as functional magnetic resonance imaging (fMRI) data in medicine systems and user-product data in product information systems. By using these data to predict the relationship between tensor features and univariate responses, data empowerment can be achieved, providing more accurate services or solutions, such as disease decision diagnosis or product recommendations. Currently available tensor regression methods, however, present two major shortcomings: the spatial information of tensors may be lost in these models, resulting in inaccurate prediction results; the calculation cost is too high, which results in untimely solutions or services. The two problems are more severe for large-scale data with high-order structures. Therefore, in order to achieve data empowerment, that is, to use tensor data to improve the quality and efficiency of information services or solutions, we propose sparse and low-rank tensor regression model (SLTR). This model enforces sparsity and low-rankness of the tensor coefficient by directly applying ${\mathscr{l}}_{1}$ norm and tensor nuclear norm on it respectively, such that not only the structural information of the tensor is preserved but also the data interpretation is convenient. To make the solving procedure scalable and efficient, SLTR makes use of the proximal gradient method to optimize the hybrid regularizer, which can be easily implemented parallelly. Additionally, a tight error bound of SLTR is theoretically proved. We evaluate SLTR on several simulated datasets and one video dataset. Experimental results show that, compared with previous models, SLTR is capable to obtain a better solution with much fewer time costs.
- tensor regression,
- parallel proximal method,
- data interpretation,
- tensor norm,
- classification

FullText(HTML)

References (29)

References

[1]	Zhu Dajiang, Zhang Tuo, Jiang Xi, et al. Fusing DTI and fMRI data: A survey of methods and applications[J]. NeuroImage, 2014, 102: 184−191 doi: 10.1016/j.neuroimage.2013.09.071
[2]	Noroozi A, Rezghi M. A tensor-based framework for rs-fMRI classification and functional connectivity construction[J]. Frontiers in Neuroinformatics, 2020, 14: 581897 doi: 10.3389/fninf.2020.581897
[3]	Wu X, Lai J. Tensor-based projection using ridge regression and its application to action classification[J]. IET Image Processing, 2010, 4(6): 486−493 doi: 10.1049/iet-ipr.2009.0278
[4]	Lui Y M. A least squares regression framework on manifolds and its application to gesture recognition[C]//Proc of 2012 IEEE Computer Society Conf on Computer Vision and Pattern Recognition Workshops. Piscataway, NJ: IEEE, 2012: 13−18
[5]	Yang Yinchong, Krompass D, Tresp V. Tensor-train recurrent neural networks for video classification[C]//Proc of the 34th Int Conf on Machine Learning. New York: PMLR, 2017: 3891−3900
[6]	Sharma L, Gera A. A survey of recommendation system: Research challenges[J]. International Journal of Engineering Trends and Technology, 2013, 4(5): 1989−1992
[7]	Bhargava P, Phan T, Zhou Jiayu, et al. Who, what, when, and where: Multi-dimensional collaborative recommendations using tensor factorization on sparse user-generated data[C]//Proc of the 24th Int Conf on World Wide Web. New York: ACM, 2015: 130−140
[8]	Mitchell T M, Shinkareva S V, Carlson A, et al. Predicting human brain activity associated with the meanings of nouns[J]. Science, 2008, 320(5880): 1191−1195 doi: 10.1126/science.1152876
[9]	Soomro K, Zamir A R, Shah M. UCF101: A dataset of 101 human actions classes from videos in the wild[J]. arXiv preprint, arXiv: 1212.0402, 2012
[10]	Huang Qingqiu, Xiong Yu, Rao Anyi, et al. MovieNet: A holistic dataset for movie understanding[C]//Proc of the 16th European Conf. Berlin: Springer, 2020: 709−727
[11]	Zhou Hua, Li Lexin, Zhu Hongtu. Tensor regression with applications in neuroimaging data analysis[J]. Journal of the American Statistical Association, 2013, 108(502): 540−552 doi: 10.1080/01621459.2013.776499
[12]	He Lifang, Chen Kun, Xu Wanwan, et al. Boosted sparse and low-rank tensor regression[J]. Advances in Neural Information Processing Systems, 2018, 31[2023-07-30].https://proceedings.neurips.cc/paper/2018/hash/8d34201a5b85900908db6cae92723617-Abstract.html
[13]	Li Na, Stefan K, Carmeliza N. Some convergence results on the regularized alternating least-squares method for tensor decomposition[J]. Linear Algebra and Its Applications, 2013, 438(2): 796−812 doi: 10.1016/j.laa.2011.12.002
[14]	Cichocki A, Lee N, Oseledets I, et al. Tensor networks for dimensionality reduction and large-scale optimization: Part 1 low-rank tensor decompositions[J]. Foundations and Trends® in Machine Learning, 2016, 9(4/5): 249−429
[15]	Song Xiaonan, Lu Haiping. Multi-linear regression for embedded feature selection with application to fMRI analysis[C/OL]//Proc of the 31st AAAI Conf on Artificial Intelligence. 2017[2023-06-30].https://ojs.aaai.org/index.php/AAAI/article/view/10871
[16]	Li Wenwen, Lou Jian, Zhou Shuo, et al. Sturm: Sparse tubal-regularized multilinear regression for fMRI[C]//Proc of 10th Int Workshop on Machine Learning in Medical Imaging. Berlin: Springer, 2019: 256−264
[17]	Yang E, Lozano A, Ravikumar P. Elementary estimators for high-dimensional linear regression[C]//Proc of the 31st Int Conf on Machine Learning. New York: PMLR, 2014: 388−396
[18]	Tucker L R. Some mathematical notes on three-mode factor analysis[J]. Psychometrika, 1966, 31(3): 279−311 doi: 10.1007/BF02289464
[19]	Rabanser S, Shchur O, Günnemann S. Introduction to tensor decompositions and their applications in machine learning[J]. arXiv preprint, arXiv: 1711.10781, 2017
[20]	Hillar C J, Lim L H. Most tensor problems are NP-hard[J]. Journal of the ACM, 2013, 60(6): 1−39
[21]	Tomioka R, Hayashi K, Kashima H. Estimation of low-rank tensors via convex optimization[J]. arXiv preprint, arXiv: 1010.0789, 2010
[22]	Liu Ji, Musialski P, Wonka P, et al. Tensor completion for estimating missing values in visual data[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2012, 35(1): 208−220
[23]	Boyd S, Parikh N, Chu E, et al. Distributed optimization and statistical learning via the alternating direction method of multipliers[J]. Foundations and Trends® in Machine Learning, 2011, 3(1): 1−122
[24]	Combettes P L, Pesquet J C. Proximal splitting methods in signal processing[J]. Fixed-point Algorithms for Inverse Problems in Science and Engineering, 2011, 49: 185−212
[25]	Negahban S N, Ravikumar P, Wainwright M J, et al. A unified framework for high-dimensional analysis of M-estimators with decomposable regularizers[J/OL]. 2012[2023-07-31].https://projecteuclid.org/journals/statistical-science/volume-27/issue-4/A-Unified-Framework-for-High-Dimensional-Analysis-of-M-Estimators/10.1214/12-STS400.full
[26]	Guo Weiwei, Kotsia I, Patras I. Tensor learning for regression[J]. IEEE Transactions on Image Processing, 2011, 21(2): 816−827
[27]	Zou Hui, Hastie T. Regularization and variable selection via the elastic net[J]. Journal of the Royal Statistical Society Series B: Statistical Methodology, 2005, 67(2): 301−320 doi: 10.1111/j.1467-9868.2005.00503.x
[28]	Panagakis Y, Kossaifi J, Chrysos G G, et al. Tensor methods in computer vision and deep learning[J]. Proceedings of the IEEE, 2021, 109(5): 863−890 doi: 10.1109/JPROC.2021.3074329
[29]	陈珂锐,孟小峰. 机器学习的可解释性[J]. 计算机研究与发展,2020,57(9):1971−1986 doi: 10.7544/issn1000-1239.2020.20190456 Chen Kerui, Meng Xiaofeng. Interpretation and understanding in machine learning[J]. Journal of Computer Research and Development, 2020, 57(9): 1971−1986 (in Chinese) doi: 10.7544/issn1000-1239.2020.20190456

[1]	Wang Chenze, Shen Xuehao, Huang Zhenli, Wang Zhengxia. Interactive Visualization Framework for Panoramic Super-Resolution Images Based on Localization Data[J]. Journal of Computer Research and Development, 2024, 61(7): 1741-1753. DOI: 10.7544/issn1000-1239.202330643
[2]	Fan Wei, Liu Yong. Social Network Information Diffusion Prediction Based on Spatial-Temporal Transformer[J]. Journal of Computer Research and Development, 2022, 59(8): 1757-1769. DOI: 10.7544/issn1000-1239.20220064
[3]	Zhou Weilin, Yang Yuan, Xu Mingwei. Network Function Virtualization Technology Research[J]. Journal of Computer Research and Development, 2018, 55(4): 675-688. DOI: 10.7544/issn1000-1239.2018.20170937
[4]	Yang Shuaifeng, Zhao Ruizhen. Image Super-Resolution Reconstruction Based on Low-Rank Matrix and Dictionary Learning[J]. Journal of Computer Research and Development, 2016, 53(4): 884-891. DOI: 10.7544/issn1000-1239.2016.20140726
[5]	Dou Nuo, Zhao Ruizhen, Cen Yigang, Hu Shaohai, Zhang Yongdong. Noisy Image Super-Resolution Reconstruction Based on Sparse Representation[J]. Journal of Computer Research and Development, 2015, 52(4): 943-951. DOI: 10.7544/issn1000-1239.2015.20140047
[6]	Yang Xin, Zhou Dake, Fei Shumin. A Self-Adapting Bilateral Total Variation Technology for Image Super-Resolution Reconstruction[J]. Journal of Computer Research and Development, 2012, 49(12): 2696-2701.
[7]	Wang Kai, Hou Zifeng. A Relaxed Co-Scheduling Method of Virtual CPUs on Xen Virtual Machines[J]. Journal of Computer Research and Development, 2012, 49(1): 118-127.
[8]	Wang Dan, Feng Dengguo, and Xu Zhen. An Approach to Data Sealing Based on Trusted Virtualization Platform[J]. Journal of Computer Research and Development, 2009, 46(8): 1325-1333.
[9]	Xiao Chuangbai, Yu Jing, Xue Yi. A Novel Fast Algorithm for MAP Super-Resolution Image Reconstruction[J]. Journal of Computer Research and Development, 2009, 46(5): 872-880.
[10]	Huang Hua, Fan Xin, Qi Chun, and Zhu Shihua. Face Image Super-Resolution Reconstruction Based on Recognition and Projection onto Convex Sets[J]. Journal of Computer Research and Development, 2005, 42(10): 1718-1725.