Abstract: Human action recognition is an important issue in the field of computer vision. Compared with object recognition in still images, human action recognition has more concerns on the spatio-temporal motion changes of interesting objects in image sequences. The extension of 2D image to 3D spatio-temporal image sequence increases the complexity of action recognition greatly, Meanwhile, it also provides a wide space for various attempts on different solutions and techniques on human action recognition. Recently, many new algorithms and systems on human action recognition have emerged, which indicates that it has become one of the hottest topics in computer vision. In this paper, we propose a taxonomy of human action recognition in chronological order to classify action recognition methods into different periods and put forward general summaries of them. Compared with other surveys, the proposed taxonomy introduces human action recognition methods and summarizes their characteristics by analyzing the action dataset evolution and responding recognition methods. Furthermore, the introduction of action recognition datasets coincides with the trend of big data-driven research idea. Through the summarization on related work, we also give some prospects on future work.