Abstract: Computing power network breaks the performance bottleneck of single network point by collaborating the computing resources of cloud servers, edge nodes and devices. Computing power network provides support for intelligent society through computing power. Computing and network convergence has gradually become the trend of new information and communication network technology. Due to heterogeneous computing resources and dynamic network, it is an challenging problem to cooperate the computing resources of cloud, edge and devices to reduce the delay of computing tasks in computing power network. To simplify the problem, existing work often assumes that the system time is discrete and only make computing offloading decisions at the end of time slots. However, this assumption results in decision waiting latency and increases the overall waiting delay of computing tasks. To address this problem, we propose a time-continuous computing task offloading optimization mechanism for computing and network convergence. Under the premise of continues timeline and collaboration of edge nodes, we model the computing offloading problem between cloud, edge and devices with the optimization goal of service experience improvement rate. Besides, we design a computing offloading algorithm based on deep reinforcement learning to improve the utility of computing resources in computing power network. Through extensive simulation experiments, it is demonstrated that the proposed algorithm can effectively reduce the task delay and improve the service experience compared with two baseline algorithms.