一种基于裸闪存的Key-Value数据库优化方法

秦雄军; 张佳程; 陆游游; 舒继武

doi:10.7544/issn1000-1239.2017.20170092

一种基于裸闪存的Key-Value数据库优化方法

(清华大学计算机科学与技术系北京 100084) (qinxiongjun2010@163.com)

基金项目: 国家自然科学基金项目(61327902,61433008)；北京市科委课题(D151100000815003)

详细信息

中图分类号: TP333
计量
- 文章访问数: 1745
- HTML全文浏览量: 3
- PDF下载量: 879
出版历程
- 发布日期: 2017-05-31

A Key-Value Database Optimization Method Based on Raw Flash Device

(Department of Computer Science and Technology, Tsinghua University, Beijing 100084)

摘要

摘要: 近年来，非关系型的key-value数据库得到越来越广泛的应用.然而，目前主流的key-value数据库或者是基于磁盘设计的，或者是传统的基于文件系统和闪存转换层FTL来构建的，难以发挥闪存存储设备的特性，限制了I/O的并发性能，且垃圾回收过程复杂.设计并实现了一种基于裸闪存的key-value数据管理架构Flashkv，通过用户态下的管理单元进行空间管理和垃圾回收，充分利用了闪存设备内部的并发特性，并简化了垃圾回收过程，去除了传统文件系统和FTL中的冗余功能，缩短了I/O路径.提出了基于闪存特点的I/O调度技术，优化了闪存的读写延迟，提高了吞吐率；提出了用户态缓存管理技术，降低了数据写入量和频繁系统调用所带来的开销.测试结果表明，Flashkv性能是levelDB的1.9~2.2倍，写入量减少60%~65%.
- key-value数据库 /
- 闪存 /
- 裸设备 /
- 数据存储 /
- 使用寿命
Abstract: In recent years, NoSQL key-value databases have been widely used. However, the current mainstream key-value databases are based either on disk, or on traditional file system and flash translation layer, which makes it difficult to utilize the characteristics of flash devices, and also limits I/O concurrency of flash devices. Moreover, garbage collection process under such kind of architecture is complex. This paper designs and implements Flashkv, a key-value data management architecture based on raw flash device. Flashkv doesn’t use file system and flash translation layer, instead, it’s space management and garbage collection are done by the management unit in the user mode. Flashkv makes full use of the concurrent features inside the flash device, and simplifies the garbage collection process and removes redundant function modules which exist in both traditional file system and flash translation layer, and also shortens the I/O path. This paper proposes I/O scheduling technology based on the characteristics of flash memory, which reduces read and write latency of flash memory and improves throughput. The user mode cache management technology is proposed, which reduces write amount and also the cost of frequent system calls. Test results show that Flashkv’s performance is 1.9 to 2.2 times that of levelDB and the write amount reduces by 60% to 65%.
- key-value database /
- flash memory /
- raw device /
- data storage /
- lifespan