Residential Collegefalse
Status已發表Published
Towards Cluster-wide Deduplication Based on Ceph
Jinpeng Wang1,2; Yang Wang1; Hekang Wang1,2; Kejiang Ye1; Chengzhong Xu1,3; Shuibing He4; Lingfang Zeng5
2019-08-01
Conference Name14th IEEE International Conference on Networking, Architecture and Storage (NAS)
Source Publication2019 IEEE International Conference on Networking, Architecture and Storage, NAS 2019 - Proceedings
Pages8834729
Conference Date15-17 August 2019
Conference PlaceEnShi, China
CountryChina
Abstract

In this paper, we design an efficient deduplication algorithm based on the distributed storage architecture of Ceph. The algorithm uses on-line block-level data deduplication technology to complete data slicing, which neither affects the data storage process in Ceph nor alter other interfaces and functions in Ceph. Without relying on any central node, the algorithm maintains the characteristics of Ceph by designing a special hash object to store the data fingerprint, and uses the CRUSH algorithm to judge the data duplication based on calculation, instead of global search. The algorithm replaces the duplicate data with the deduplicated objects, which storage their fingerprints with less storage space. We compare the effects of different block sizes with respect to the performance and deduplication rates through experimental studies, and select the most appropriate block size in our prototype implementation. The experimental results show that the algorithm can not only effectively save the storage space but also improve the bandwidth utilization when reading and writing the duplicate data.

KeywordCeph Deduplication Distributed Storage System
DOI10.1109/NAS.2019.8834729
URLView the original
Indexed ByCPCI-S
Language英語English
WOS Research AreaComputer Science ; Telecommunications
WOS SubjectComputer Science, Hardware & Architecture ; Telecommunications
WOS IDWOS:000589508200014
The Source to Articlehttps://ieeexplore.ieee.org/document/8834729
Scopus ID2-s2.0-85073246393
Fulltext Access
Citation statistics
Document TypeConference paper
CollectionDEPARTMENT OF COMPUTER AND INFORMATION SCIENCE
Faculty of Science and Technology
Corresponding AuthorYang Wang
Affiliation1.Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences, China
2.University of Science and Technology of China
3.University of Macau, China
4.Zhejiang University, China
5.Huazhong University of Science and Technology, China
Recommended Citation
GB/T 7714
Jinpeng Wang,Yang Wang,Hekang Wang,et al. Towards Cluster-wide Deduplication Based on Ceph[C], 2019, 8834729.
APA Jinpeng Wang., Yang Wang., Hekang Wang., Kejiang Ye., Chengzhong Xu., Shuibing He., & Lingfang Zeng (2019). Towards Cluster-wide Deduplication Based on Ceph. 2019 IEEE International Conference on Networking, Architecture and Storage, NAS 2019 - Proceedings, 8834729.
Files in This Item:
There are no files associated with this item.
Related Services
Recommend this item
Bookmark
Usage statistics
Export to Endnote
Google Scholar
Similar articles in Google Scholar
[Jinpeng Wang]'s Articles
[Yang Wang]'s Articles
[Hekang Wang]'s Articles
Baidu academic
Similar articles in Baidu academic
[Jinpeng Wang]'s Articles
[Yang Wang]'s Articles
[Hekang Wang]'s Articles
Bing Scholar
Similar articles in Bing Scholar
[Jinpeng Wang]'s Articles
[Yang Wang]'s Articles
[Hekang Wang]'s Articles
Terms of Use
No data!
Social Bookmark/Share
All comments (0)
No comment.
 

Items in the repository are protected by copyright, with all rights reserved, unless otherwise indicated.