Residential Collegefalse
Status已發表Published
Multi Resource Scheduling with Task Cloning in Heterogeneous Clusters
Xu HL(徐歡樂)1; Yang Liu2; Wing Cheong Lau3
2022-06
Conference NameInternational Conference on Parallel Processing
Source PublicationICPP 2022 - Proceedings of the 2022 International Conference on Parallel Processing
Pages1-11
Conference Date2022-8-29
Conference PlaceInria
CountryFrance
PublisherAssociation for Computing Machinery
Abstract

To mitigate the straggler effect, today's systems and computing frameworks have adopted redundancy to launch extra copies for stragglers. Two limitations of the existing straggler-mitigation techniques, however, are that resource demand of tasks is only considered in the context of slots and, moreover, redundancy is seldom coordinated with job scheduling. To tackle these issues, in this paper, we present DollyMP, a job scheduler that addresses multi-resource scheduling with task cloning in heterogeneous clusters. DollyMP carefully combines SRPT (Shortest Remaining Processing Time) and SVF (Smallest Volume First) via knapsack optimization to schedule tasks with multi-resource demands and, in the meanwhile, dynamically launches task clones to yield a small job completion time. DollyMP is built on a strong mathematical foundation to guarantee near-optimal performance. The deployment of our Hadoop YARN prototype on a 30-node cluster demonstrates that DollyMP can reduce job response time by 50% under different cluster loads.

DOI10.1145/3545008.3545093
Language英語English
Scopus ID2-s2.0-85148620520
Fulltext Access
Citation statistics
Document TypeConference paper
CollectionDEPARTMENT OF COMPUTER AND INFORMATION SCIENCE
Affiliation1.University of Macau
2.Shanghai University
3.Chinese University of Hong Kong
First Author AffilicationUniversity of Macau
Recommended Citation
GB/T 7714
Xu HL,Yang Liu,Wing Cheong Lau. Multi Resource Scheduling with Task Cloning in Heterogeneous Clusters[C]:Association for Computing Machinery, 2022, 1-11.
APA Xu HL., Yang Liu., & Wing Cheong Lau (2022). Multi Resource Scheduling with Task Cloning in Heterogeneous Clusters. ICPP 2022 - Proceedings of the 2022 International Conference on Parallel Processing, 1-11.
Files in This Item: Download All
File Name/Size Publications Version Access License
DollyMP-camera-ready(2072KB)会议论文 开放获取CC BY-NC-SAView Download
Related Services
Recommend this item
Bookmark
Usage statistics
Export to Endnote
Google Scholar
Similar articles in Google Scholar
[Xu HL(徐歡樂)]'s Articles
[Yang Liu]'s Articles
[Wing Cheong Lau]'s Articles
Baidu academic
Similar articles in Baidu academic
[Xu HL(徐歡樂)]'s Articles
[Yang Liu]'s Articles
[Wing Cheong Lau]'s Articles
Bing Scholar
Similar articles in Bing Scholar
[Xu HL(徐歡樂)]'s Articles
[Yang Liu]'s Articles
[Wing Cheong Lau]'s Articles
Terms of Use
No data!
Social Bookmark/Share
File name: DollyMP-camera-ready.pdf
Format: Adobe PDF
All comments (0)
No comment.
 

Items in the repository are protected by copyright, with all rights reserved, unless otherwise indicated.