Abstract: With the rapid development of technology, the number of network edge devices with the capability of computation and memory is increasing, and the volume of the generated data is growing exponentially, which makes it difficult for a centralized processing model with cloud computing as the core to efficiently process data generated by edge devices. Not only will the network delay increase, but the data is likely to be leaked on the upload link, and data security cannot be guaranteed. In addition, due to the diversity of edge devices and the continuous enrichment of data representation methods, multi-modal data exists widely. The processing of multi-source heterogeneous data collected by different edge devices has become an urgent problem in big data research. In order to make full use of heterogeneous data on edge devices and solve the problem of “data communication barriers” caused by data privacy in edge computing, in this paper we propose a novel fusion algorithm for multi-source heterogeneous data based on Tucker decomposition in federated learning. For the fusion problem of heterogeneous data without interaction in federated learning, the proposed algorithm introduces Tucker decomposition theory to capture the multi-dimensional characteristics of heterogeneous data by constructing a high-order tensor. Finally, the effectiveness of this algorithm is verified on the MOSI dataset.