Quality of Service Guaranty Technology of Multi-Tenant Solid-State Drives: A Survey

Wen Yuhong; Zhou You; Wu Qiulin; Wu Fei; Xie Changsheng

doi:10.7544/issn1000-1239.202220561

Journal of Computer Research and Development > 2023 > 60(3): 555-571. > DOI: 10.7544/issn1000-1239.202220561

Wen Yuhong, Zhou You, Wu Qiulin, Wu Fei, Xie Changsheng. Quality of Service Guaranty Technology of Multi-Tenant Solid-State Drives: A Survey[J]. Journal of Computer Research and Development, 2023, 60(3): 555-571. DOI: 10.7544/issn1000-1239.202220561

Citation:

PDF (2010 KB)

Quality of Service Guaranty Technology of Multi-Tenant Solid-State Drives: A Survey

Wen Yuhong^{1, 2,},
Zhou You^2, ,,
Wu Qiulin¹,
Wu Fei^1,,
Xie Changsheng¹

1.
Wuhan National Laboratory for Optoelectronics(Huazhong University of Science and Technology), Wuhan 430074
2.
School of Computer Science and Technology, Huazhong University of Science and Technology, Wuhan 430074

Funds: This work was supported by the National Natural Science Foundation of China (U2001203, 61902137, 61821003, 61872413) and the Fundamental Research Funds for the Central Universities (2021XXJS108, YCJJ202202002).

More Information

Received Date: June 15, 2022
Revised Date: December 26, 2022
Available Online: February 26, 2023

Graphical Abstract

Abstract

Abstract

Sharing a solid-state drive (SSD) among multiple tenants has become a common way to improve the storage resource utilization and reduce the operating cost. This is driven by the high-density flash technology development, highly parallel storage architecture inside SSDs, and the full support of interface techniques. However, multiple tenants contend for the limited storage resources including the time resource and space resource of the SSD and interfere with each other. How to guarantee the quality of service (QoS) of multi-tenant SSDs has been a hot research topic in recent years. In this paper, we analyze and summarize the current state of research about the QoS technologies of multi-tenant SSDs. Firstly, we analyze the three QoS problems of multi-tenant SSDs: performance interference among tenants, performance unfairness among tenants, and loss of overall performance. Then, we present a comprehensive classification and introduction to existing work for improving the QoS of multi-tenant SSDs from three types of objectives, i.e., performance isolation, fairness guarantee, and overall performance optimization, from two dimensions of time resource and space resource. The technical evolution line of existing work is also sorted out. Finally, we conclude the existing techniques for QoS assurance of multi-tenant SSDs and discuss potential future research directions.
- flash memory,
- solid-state drive,
- multi-tenant,
- storage quality of service,
- performance isolation,
- fairness guarantee

FullText(HTML)

References (63)

References

[1]	IDC. Data age 2025 [EB/OL]. [2022-05-06].https://www.import.io/wp-content/uploads/2017/04/Seagate-WP-DataAge2025-March-2017.pdf
[2]	DapuStor. DapuStor enterprise NVMe SSD – Haishen3 [EB/OL]. [2022-05-06].https://en.dapustor.com/prod_view.aspx?nid=3&typeid=66&id=185
[3]	Memblaze. PBlaze^®6 6530 series NVMe™ SSD [EB/OL]. [2022-05-06]. http://www.memblaze.com/en/index.php?c=article&a=type&tid=111
[4]	Samsung. Samsung electronics debuts industry-leading 8TB consumer SSD, the 870 QVO [EB/OL]. [2022-05-06].https://news.samsung.com/global/samsung-electronics-debuts-industry-leading-8tb-consumer-ssd-the-870-qvo
[5]	Solidigm. D5-P5316 SSD [EB/OL]. [2022-05-06].https://www.solidigmtech.com.cn/content/solidigm/us/en/products/data-center/d5/p5316.html
[6]	Wikibon. Flash-native architectures power next-generation real-time workloads [EB/OL]. [2022-05-06].https://wikibon.com/flash-native-drives-real-time-business-process/
[7]	陆游游,舒继武. 闪存存储系统综述[J]. 计算机研究与发展,2013,50(1):49−59 doi: 10.7544/issn1000-1239.2013.20120699 Lu Youyou, Shu Jiwu. Survey on flash-based storage systems[J]. Journal of Computer Research and Development, 2013, 50(1): 49−59 (in Chinese) doi: 10.7544/issn1000-1239.2013.20120699
[8]	舒继武,陆游游,张佳程,等. 基于非易失性存储器的存储系统技术研究进展[J]. 科技导报,2016,34(14):86−94 Shu Jiwu, Lu Youyou, Zhang Jiacheng, et al. Research progress on non-volatile memory based storage system[J]. Science & Technology Review, 2016, 34(14): 86−94 (in Chinese)
[9]	高聪明,石亮,刘凯,等. 闪存固态硬盘系统结构与技术[J]. 计算机研究与发展,2021,58(7):1518−1532 doi: 10.7544/issn1000-1239.2021.20200690 Gao Congming, Shi Liang, Liu Kai, et al. Architecture and technologies of flash memory based solid state drives[J]. Journal of Computer Research and Development, 2021, 58(7): 1518−1532 (in Chinese) doi: 10.7544/issn1000-1239.2021.20200690
[10]	Kim J, Lee D, Noh S H. Towards SLO complying SSDs through OPS isolation[C]//Proc of the 13th USENIX Conf on File and Storage Technologies (FAST’15). Berkeley, CA: USENIX Association, 2015: 183−189
[11]	Tavakkol A, Sadrosadati M, Ghose S, et al. FLIN: Enabling fairness and enhancing performance in modern NVMe solid state drives[C] //Proc of the 45th ACM/IEEE Annual Int Symp on Computer Architecture (ISCA). Piscataway, NJ: IEEE, 2018: 397−410
[12]	Huang Jian, Badam A, Caulfield L, et al. FlashBlox: Achieving both performance isolation and uniform lifetime for virtualized SSDs[C] //Proc of the 15th USENIX Conf on File and Storage Technologies (FAST’17). Berkeley, CA: USENIX Association, 2017: 375−390
[13]	Kwon M, Gouk D, Lee C, et al. DC-Store: Eliminating noisy neighbor containers using deterministic I/O performance and resource isolation[C] //Proc of the 18th USENIX Conf on File and Storage Technologies (FAST’20). Berkeley, CA: USENIX Association, 2020: 183−191
[14]	Birke R, Bjoerkqvist M, Chen L Y, et al. (Big) Data in a virtualized world: Volume, velocity, and variety in cloud datacenters[C] //Proc of the 12th USENIX Conf on File and Storage Technologies (FAST’14). Berkeley, CA: USENIX Association, 2014: 177−189
[15]	Ding Zhimin. I/O virtualization in enterprise SSDs [C/OL] //Proc of the Storage Developer Conf. Santa Clara, CA: Storage Networking Industry Association, 2015[2022-05-06].https://www.snia.org/sites/default/files/SDC15_presentations/virt/ZhiminDing_IO_Virtualization_eSSD.pdf
[16]	NVM Express. The NVMe 2.0 specification [EB/OL]. [2022-05-06]. https://nvmexpress.org/developers/
[17]	Peng Bo, Zhang Haozhong, Yao Jianguo, et al. MDev-NVMe: A NVMe storage virtualization solution with mediated pass-through[C] //Proc of the USENIX Annual Technical Conf (USENIX ATC’18). Berkeley, CA: USENIX Association, 2018: 665−676
[18]	ONFI. Open NAND flash interface specification [EB/OL]. [2022-05-06].https://media-www.micron.com/-/media/client/onfi/specs/onfi_4_2-gold.pdf
[19]	Hu Yang, Jiang Hong, Feng Dan, et al. Performance impact and interplay of SSD parallelism through advanced commands, allocation strategy and data granularity[C] //Proc of the Int Conf on Supercomputing. New York: ACM, 2011: 96−107
[20]	Nam E H, Kim B S J, Eom H, et al. Ozone (O3): An out-of-order flash memory controller architecture[J]. IEEE Transactions on Computers, 2010, 60(5): 653−666
[21]	Elyasi N, Arjomand M, Sivasubramaniam A, et al. Exploiting intra-request slack to improve SSD performance[C] //Proc of the 22nd Int Conf on Architectural Support for Programming Languages and Operating Systems. New York: ACM, 2017: 375−388
[22]	Gupta A, Kim Y, Urgaonkar B. DFTL: A flash translation layer employing demand-based selective caching of page-level address mappings[J]. ACM SIGPLAN Notices, 2009, 44(3): 229−240 doi: 10.1145/1508284.1508271
[23]	Kang J, Jo H, Kim J, et al. A superblock-based flash translation layer for NAND flash memory[C] //Proc of the 6th ACM & IEEE Int Conf on Embedded Software. New York: ACM, 2006: 161−170
[24]	Lin W H, Chang Lipin. Dual greedy: Adaptive garbage collection for page-mapping solid-state disks[C] //Proc of the Design, Automation & Test in Europe Conf & Exhibition (DATE). Piscataway, NJ: IEEE, 2012: 117−122
[25]	Debnath B, Krishnan S, Xiao Weijun, et al. Sampling-based garbage collection metadata management scheme for flash-based storage[C/OL] //Proc of the 27th IEEE Symp on Mass Storage Systems and Technologies (MSST). Piscataway, NJ: IEEE, 2011[2022-05-06].https://ieeexplore.ieee.org/abstract/document/5937228/
[26]	Chang Yuanhao, Hsieh J W, Kuo Teiwei. Improving flash wear-leveling by proactively moving static data[J]. IEEE Transactions on Computers, 2009, 59(1): 53−65
[27]	Murugan M, Du D H C. Rejuvenator: A static wear leveling algorithm for NAND flash memory with minimized overhead[C/OL] //Proc of the 27th IEEE Symp on Mass Storage Systems and Technologies (MSST). Piscataway, NJ: IEEE, 2011[2022-05-06].https://ieeexplore.ieee.org/abstract/document/5937225/
[28]	Wu Guanying, He Xubin. Reducing SSD read latency via NAND flash program and erase suspension[C/OL] //Proc of the 10th USENIX Conf on File and Storage Technologies (FAST’12). Berkeley, CA: USENIX Association, 2012[2022-05-06]. https://www.usenix.org/legacy/event/fast12/tech/full_papers/Wu.pdf
[29]	Hu Xiaoyu, Eleftheriou E, Haas R, et al. Write amplification analysis in flash-based solid state drives[C/OL] //Proc of the Israeli Experimental Systems Conf. New York: ACM, 2009[2022-05-06].https://dl.acm.org/doi/abs/10.1145/1534530.1534544
[30]	Kim J, Lee E, Noh S H. I/O scheduling schemes for better I/O proportionality on flash-based SSDs[C] //Proc of the 24th IEEE Int Symp on Modeling, Analysis and Simulation of Computer and Telecommunication Systems (MASCOTS). Piscataway, NJ: IEEE, 2016: 221−230
[31]	Park S, Shen Kai. FIOS: A fair, efficient flash I/O scheduler[C/OL]//Proc of the 10th USENIX Conf on File and Storage Technologies (FAST’12). Berkeley, CA: USENIX Association, 2012[2022-05-06].https://www.usenix.org/legacy/event/fast12/tech/full_papers/Park.pdf
[32]	Kim B S. Utilitarian performance isolation in shared SSDs[C/OL] //Proc of the 10th USENIX Workshop on Hot Topics in Storage and File Systems (HotStorage’18). Berkeley, CA: USENIX Association, 2018[2022-05-06].https://www.usenix.org/system/files/conference/hotstorage18/hotstorage18-paper-kim-bryan.pdf
[33]	Liu Renping, Liu Duo, Chen Xianzhang, et al. Self-adapting channel allocation for multiple tenants sharing SSD devices[J]. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, 2021, 41(2): 294−305
[34]	Yang Mingchang, Chang Yuanhao, Tsao C W, et al. Utilization-aware self-tuning design for TLC flash storage devices[J]. IEEE Transactions on Very Large Scale Integration (VLSI) Systems, 2016, 24(10): 3132−3144 doi: 10.1109/TVLSI.2016.2538182
[35]	Wu Guanying, He Xubin. Reducing SSD read latency via NAND flash program and erase suspension[C/OL] //Proc of the 10th USENIX Conf on File and Storage Technologies (FAST’12). Berkeley, CA: USENIX Association, 2012[2022-05-06].https://www.usenix.org/legacy/event/fast12/tech/full_papers/Wu.pdf
[36]	Paul M. CGROUPS [EB/OL]. [2022-05-06].https://docs.kernel.org/admin-guide/cgroup-v1/cgroups.html
[37]	Axboe J. Linux block IO—Present and future[C/OL] //Proc of the Ottawa Linux Symp. 2004: 51−61[2022-05-06]. http://www.mnis.fr/fr/services/virtualisation/pdf/LinuxSymposium2004_V1.pdf#page=51
[38]	Valente P, Avanzini A. Evolution of the BFQ storage-I/O scheduler[C] // Proc of the Mobile Systems Technologies Workshop (MST). Piscataway, NJ: IEEE, 2015: 15−20
[39]	Kim J, Kim D, Won Y. Fair I/O scheduler for alleviating read/write interference by forced unit access in flash memory[C] //Proc of the 14th ACM Workshop on Hot Topics in Storage and File Systems. Berkeley, CA: USENIX Association, 2022: 86−92
[40]	Choi W, Urgaonkar B, Kandemir M, et al. Fair write attribution and allocation for consolidated flash cache[C] //Proc of the 25th Int Conf on Architectural Support for Programming Languages and Operating Systems. New York: ACM, 2020: 1063−1076
[41]	Zhang Quan, Feng Dan, Wang Fang, et al. An efficient, QoS-aware I/O scheduler for solid state drive[C] //Proc of the 10th IEEE Int Conf on High Performance Computing and Communications & 2013 IEEE Int Conf on Embedded and Ubiquitous Computing. Piscataway, NJ: IEEE, 2013: 1408−1415
[42]	Jun B, Shin D. Workload-aware budget compensation scheduling for NVMe solid state drives[C/OL] //Proc of the IEEE Non-Volatile Memory System and Applications Symp (NVMSA). Piscataway, NJ: IEEE, 2015 [2022-05-06]. https://ieeexplore.ieee.org/abstract/document/7304369
[43]	Chang Dawei, Chen H H, Su Weijian. VSSD: Performance isolation in a solid-state drive[J]. ACM Transactions on Design Automation of Electronic Systems, 2015, 20(4): 1−33
[44]	Nowak A S, Radzik T. The Shapley value for n-person games in generalized characteristic function form[J]. Games and Economic Behavior, 1994, 6(1): 150−161 doi: 10.1006/game.1994.1008
[45]	Joshi K, Yadav K, Choudhary P. Enabling NVMe WRR support in Linux block layer[C/OL] //Proc of the 9th USENIX Workshop on Hot Topics in Storage and File Systems (HotStorage’17). Berkeley, CA: USENIX Association, 2017[2022-05-06].https://www.usenix.org/conference/hotstorage17/program/presentation/joshi
[46]	Shreedhar M, Varghese G. Efficient fair queueing using deficit round robin[C] //Proc of the Conf on Applications, Technologies, Architectures, and Protocols for Computer Communication. New York: ACM, 1995: 231−242
[47]	Goyal P, Vin H M, Chen Haichen. Start-time fair queueing: A scheduling algorithm for integrated services packet switching networks[C] //Proc of the Conf on Applications, Technologies, Architectures, and Protocols for Computer Communications. New York: ACM, 1996: 157−168
[48]	Jin Wei, Chase J S, Kaur J. Interposed proportional sharing for a storage service utility[J]. ACM SIGMETRICS Performance Evaluation Review, 2004, 32(1): 37−48 doi: 10.1145/1012888.1005694
[49]	Shen Kai, Park S. FlashFQ: A fair queueing I/O scheduler for flash-based SSDs[C] // Proc of the USENIX Annual Technical Conf (USENIX ATC’13). Berkeley, CA: USENIX Association, 2013: 67−78
[50]	Hedayati M, Shen Kai, Scott M L, et al. Multi-queue fair queuing[C] //Proc of the USENIX Annual Technical Conf (USENIX ATC’19). Berkeley, CA: USENIX Association, 2019: 301−314
[51]	Woo J, Ahn M, Lee G, et al. D2FQ: Device-direct fair queueing for NVMe SSDs[C] //Proc of the 19th USENIX Conf on File and Storage Technologies (FAST’21). Berkeley, CA: USENIX Association, 2021: 403−415
[52]	Bjørling M, Axboe J, Nellans D, et al. Linux block IO: Introducing multi-queue SSD access on multi-core systems[C/OL] //Proc of the 6th Int Systems and Storage Conf. New York: ACM, 2013[2022-05-06].https://dl.acm.org/doi/abs/10.1145/2485732.2485740
[53]	SanDisk. Skyhawk & Skyhawk ultra NVMe PCIe SSD [EB/OL]. [2022-05-06].https://www.sandisk.com/content/dam/sandisk-main/en_us/assets/resources/data-sheets/Skyhawk-Series-NVMe-PCIe-SSD-DS.pdf
[54]	Liu Renping, Tan Zhenhua, Long Linbo, et al. Improving fairness for SSD devices through DRAM over-provisioning cache management[J]. IEEE Transactions on Parallel and Distributed Systems, 2022, 33(10): 2444−2454 doi: 10.1109/TPDS.2022.3143295
[55]	Park J, Eom Y I. Weight-aware cache for application-level proportional I/O sharing[J]. IEEE Transactions on Computers, 2021, 71(10): 2395−2407
[56]	Min D, Kim Y. Isolating namespace and performance in key-value SSDs for multi-tenant environments[C] //Proc of the 13th ACM Workshop on Hot Topics in Storage and File Systems. New York: ACM, 2021: 8−13
[57]	Song Xiang, Yang Jian, Chen Haibo. Architecting flash-based solid-state drive for high-performance I/O virtualization[J]. IEEE Computer Architecture Letters, 2013, 13(2): 61−64
[58]	Zhou You, Wu Fei, Huang Weizhou, et al. LiveSSD: A low-interference RAID scheme for hardware virtualized SSDs[J]. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, 2020, 40(7): 1354−1366
[59]	Liu Jiahao, Wang Fang, Feng Dan. CostPI: Cost-effective performance isolation for shared NVMe SSDs[C/OL] //Proc of the 48th Int Conf on Parallel Processing. New York: ACM, 2019[2022-05-06].https://dl.acm.org/doi/abs/10.1145/3337821.3337879
[60]	Hu Yang, Jiang Hong, Feng Dan, et al. Exploring and exploiting the multilevel parallelism inside SSDs for improved performance and endurance[J]. IEEE Transactions on Computers, 2012, 62(6): 1141−1155
[61]	Github. Filebench [CP/OL]. [2022-05-06]. https://github.com/filebench/filebench/wiki.
[62]	Ahn S, La K, Kim J. Improving I/O resource sharing of Linux Cgroup for NVMe SSDs on multi-core systems[C/OL] //Proc of the 8th USENIX Workshop on Hot Topics in Storage and File Systems (HotStorage’16). Berkeley, CA: USENIX Association, 2016[2022-05-06].https://www.usenix.org/conference/hotstorage16/workshop-program/presentation/ahn
[63]	Matias B, Abutalib A, Hans H, et al. ZNS: Avoiding the block interface tax for flash-based SSDs[C] //Proc of the USENIX Annual Technical Conf (USENIX ATC’21). Berkeley, CA: USENIX Association, 2021: 689–703

[1]	Guo Jing, Hu Cunchen, Bao Yungang. A Dynamic Resource Allocation Method for High-Density Colocation Scenario[J]. Journal of Computer Research and Development, 2024, 61(9): 2384-2399. DOI: 10.7544/issn1000-1239.202221043
[2]	Guo Jing, Hu Cunchen, Bao Yungang. Survey on Guaranteeing the Performance of Co-Located Applications[J]. Journal of Computer Research and Development, 2024, 61(1): 43-65. DOI: 10.7544/issn1000-1239.202220333
[3]	Fan Zhuoya, Meng Xiaofeng. Algorithmic Fairness and Fairness Computing[J]. Journal of Computer Research and Development, 2023, 60(9): 2048-2066. DOI: 10.7544/issn1000-1239.202220625
[4]	Fu Yao, Li Qingdan, Zhang Zehui, Gao Tiegang. Data Integrity Verification Scheme for Privacy Protection and Fair Payment[J]. Journal of Computer Research and Development, 2022, 59(6): 1343-1355. DOI: 10.7544/issn1000-1239.20210023
[5]	Chen Jinyin, Chen Yipeng, Chen Yiming, Zheng Haibin, Ji Shouling, Shi Jie, Cheng Yao. Fairness Research on Deep Learning[J]. Journal of Computer Research and Development, 2021, 58(2): 264-280. DOI: 10.7544/issn1000-1239.2021.20200758
[6]	Li Jie, Zhang Jing, Li Weidong, Zhang Xuejie. A Fair Distribution Strategy Based on Shared Fair and Time-Varying Resource Demand[J]. Journal of Computer Research and Development, 2019, 56(7): 1534-1544. DOI: 10.7544/issn1000-1239.2019.20180798
[7]	Yang Tan, Feng Xiang, Yu Huiqun. Feature Selection Algorithm Based on the Multi-Colony Fairness Model[J]. Journal of Computer Research and Development, 2015, 52(8): 1742-1756. DOI: 10.7544/issn1000-1239.2015.20150245
[8]	Wang Jinhai, Huang Chuanhe, Wang Jing, He Kai, Shi Jiaoli, Chen Xi. A Heterogeneous Cloud Computing Architecture and Multi-Resource-Joint Fairness Allocation Strategy[J]. Journal of Computer Research and Development, 2015, 52(6): 1288-1302. DOI: 10.7544/issn1000-1239.2015.20150168
[9]	Gu Lianchao, Cui Lizhen. A Scalable and Self-Adjust Multi-Tenant Data Storage Strategy Under Different SLAs[J]. Journal of Computer Research and Development, 2014, 51(9): 2058-2069. DOI: 10.7544/issn1000-1239.2014.20131339
[10]	Chen Ting, Li Jiandong, Zhong Shaobo, Li Changle. A Fair-Oriented Two-Level Scheduling Scheme for QoS Guarantee in WiMAX[J]. Journal of Computer Research and Development, 2009, 46(7): 1094-1101.

Cited By

Cited by

Periodical cited type(2)

1.	胡海华，沈芙辉，童宇，赵娜. 闪存芯片的取证难题：从可靠性角度分析和优化. 中国人民公安大学学报(自然科学版). 2025(01): 89-94 .
2.	王今雨，安健，王龙翔，唐新龙，丁跃，陈睿佳. NAND闪存可靠性验证实验平台设计与应用. 实验技术与管理. 2024(02): 186-192 .