A Hierarchically Heterogeneous Federated Learning Method for Cloud-Edge-End System

Federated learning (FL) has gained applause with great advantages by replacing data transmission with uploading model parameters, effectively avoiding privacy leakage. However, when federated learning is applied to the cloud-edge-end system, for one thing, there are two-level distributed frameworks in cloud-edge-end system, i.e. edge and end, challenging the traditional single-layer federated learning; for another thing, the upper complexity bound of end models is not always the same due to resource heterogeneity, which cannot satisfy the assumption of a unified model for clients in the federated learning framework. To solve the first problem above, based on the traditional single-layer federated learning method, we propose a hierarchically federated learning method for the cloud-edge-end system. For the second problem, we split a deep model into a series of sub-models by inserting early exit branches, where sub-models with different computing complexities match different clients’ resource statuses, thus accomplishing hierarchically heterogeneous federated learning. Additionally, due to factors like human resources, there is a lot of unlabeled data on end devices, which cannot be used for effective model training. Therefore, we propose a semi-supervised learning method for federated learning to effectively utilize unlabeled data. Finally, the methods of this paper are verified by MNIST and FashionMNIST datasets. Experimental results show that compared with other methods, the method proposed in this paper can improve accuracy by up to 22% under the premise of privacy security, and at the same time, the resources overhead in computing, communication, and storage have been significantly reduced.