ISSN 1000-1239 CN 11-1777/TP

Journal of Computer Research and Development ›› 2017, Vol. 54 ›› Issue (2): 235-247.doi: 10.7544/issn1000-1239.2017.20160847

Special Issue: 2017科学大数据管理专题

Previous Articles     Next Articles

Scientific Big Data Management: Concepts, Technologies and System

Li Jianhui1, Shen Zhihong1, Meng Xiaofeng2   

  1. 1(Computer Network Information Center, Chinese Academy of Sciences, Beijing 100190);2(School of Information, Remin University of China, Beijing 100872)
  • Online:2017-02-01

Abstract: In recent years, as more and more large-scale scientific facilities have been built and significant scientific experiments have been carried out, scientific research has entered an unprecedented big data era. Scientific research in big data era is a process of big science, big demand, big data, big computing, and big discovery. It is of important significance to develop a full life cycle data management system for scientific big data. In this paper, we first introduce the background of the development of scientific big data management system. Then we specify the concepts and three key characteristics of scientific big data. After an review of scientific data resource development projects and scientific data management systems, a framework is proposed aiming at the full life cycle management of scientific big data. Further, we introduce the key technologies of the management framework including data fusion, real-time analysis, long termstorage, cloud service, and data opening and sharing. Finally, we summarize the research progress in this field, and look into the application prospects of scientific big data management system.

Key words: scientific data, big data, data pipeline, full life cycle of data

CLC Number: