Residential College | false |
Status | 已發表Published |
Towards Cluster-wide Deduplication Based on Ceph | |
Jinpeng Wang1,2; Yang Wang1; Hekang Wang1,2; Kejiang Ye1; Chengzhong Xu1,3; Shuibing He4; Lingfang Zeng5 | |
2019-08-01 | |
Conference Name | 14th IEEE International Conference on Networking, Architecture and Storage (NAS) |
Source Publication | 2019 IEEE International Conference on Networking, Architecture and Storage, NAS 2019 - Proceedings |
Pages | 8834729 |
Conference Date | 15-17 August 2019 |
Conference Place | EnShi, China |
Country | China |
Abstract | In this paper, we design an efficient deduplication algorithm based on the distributed storage architecture of Ceph. The algorithm uses on-line block-level data deduplication technology to complete data slicing, which neither affects the data storage process in Ceph nor alter other interfaces and functions in Ceph. Without relying on any central node, the algorithm maintains the characteristics of Ceph by designing a special hash object to store the data fingerprint, and uses the CRUSH algorithm to judge the data duplication based on calculation, instead of global search. The algorithm replaces the duplicate data with the deduplicated objects, which storage their fingerprints with less storage space. We compare the effects of different block sizes with respect to the performance and deduplication rates through experimental studies, and select the most appropriate block size in our prototype implementation. The experimental results show that the algorithm can not only effectively save the storage space but also improve the bandwidth utilization when reading and writing the duplicate data. |
Keyword | Ceph Deduplication Distributed Storage System |
DOI | 10.1109/NAS.2019.8834729 |
URL | View the original |
Indexed By | CPCI-S |
Language | 英語English |
WOS Research Area | Computer Science ; Telecommunications |
WOS Subject | Computer Science, Hardware & Architecture ; Telecommunications |
WOS ID | WOS:000589508200014 |
The Source to Article | https://ieeexplore.ieee.org/document/8834729 |
Scopus ID | 2-s2.0-85073246393 |
Fulltext Access | |
Citation statistics | |
Document Type | Conference paper |
Collection | DEPARTMENT OF COMPUTER AND INFORMATION SCIENCE Faculty of Science and Technology |
Corresponding Author | Yang Wang |
Affiliation | 1.Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences, China 2.University of Science and Technology of China 3.University of Macau, China 4.Zhejiang University, China 5.Huazhong University of Science and Technology, China |
Recommended Citation GB/T 7714 | Jinpeng Wang,Yang Wang,Hekang Wang,et al. Towards Cluster-wide Deduplication Based on Ceph[C], 2019, 8834729. |
APA | Jinpeng Wang., Yang Wang., Hekang Wang., Kejiang Ye., Chengzhong Xu., Shuibing He., & Lingfang Zeng (2019). Towards Cluster-wide Deduplication Based on Ceph. 2019 IEEE International Conference on Networking, Architecture and Storage, NAS 2019 - Proceedings, 8834729. |
Files in This Item: | There are no files associated with this item. |
Items in the repository are protected by copyright, with all rights reserved, unless otherwise indicated.
Edit Comment