Abstract: By deploying communication, computing and storage resources on the edge devices, mobile edge computing (MEC) can effectively overcome the problems of long transmission distance and high response delay of traditional cloud computing. Therefore, MEC can satisfy the service requirements of emerging computation-intensive and delay-sensitive applications. Nevertheless, the resources of edge devices are limited and the workload among multiple edge devices is unbalanced in MEC. In order to address the above problems, multi-edge device collaboration becomes an inevitable trend. However, multi-edge device collaboration faces two challenges. First, task offloading and service caching are mutually coupled. Second, the workload and resource state of the edge devices have the characteristics of spatial-temporal change. The two challenges significantly increase the difficulty of solving this issue. In response to the above challenges, this paper proposes the online joint optimizing mechanism of task offloading and service caching for multi-edge device collaboration. And we decouple the joint optimizing problem into two sub-problems of service caching and task offloading in this paper. For the service caching sub-problem, a collaborative service caching algorithm based on contextual combinatorial multi-armed bandit is proposed. For the task offloading sub-problem, a preference-based double-side matching algorithm is designed. Simulation results demonstrate that the proposed algorithm in this paper can efficiently reduce the overall execution delay of tasks, and realize workload balancing among edge devices.