Abstract: Computer networks have been playing an important role in modern society. The rapid growth of the network scale makes the network traffic more and more complicated, which is hard to accurately model. This condition makes the optimal routing policy in communication networks an NP-hard problem. To solve this problem, traditional methods for routing and traffic engineering mainly use hand-crafted algorithms, which cannot ensure both the accuracy and efficiency. In recent years, deep reinforcement learning (DRL)-based network routing strategies have been proposed, which overcome the shortcomings of manually analysis and modelling by human experts to some extent. However, current DRL-based routing strategies all have problems in scalability, which means they cannot be used in large scale networks. Under this circumstance, this paper proposes Hierar-DRL, a DRL-based network routing technology that employs pinning control theory. Pinning control helps Hierar-DRL to select a subset of network nodes as the target control nodes of DRL. With the advantages of pinning control and the automatic policy exploring ability of DRL, Hierar-DRL shows better scalability in large networks. Simulation results show that the proposed scheme can reduce the average end-to-end transmission delay in the test network topologies by up to 28.5% compared with the state-of-the-art, which validates the proposed scheme.