Residential College | false |
Status | 已發表Published |
Understanding and Optimizing Workloads for Unified Resource Management in Large Cloud Platforms | |
Lu, Chengzhi1; Xu, Huanle2; Ye, Kejiang3; Xu, Guoyao4; Zhang, Liping4; Yang, Guodong4; Xu, Chengzhong2 | |
2023-05-08 | |
Conference Name | 18th European Conference on Computer Systems, EuroSys 2023 |
Source Publication | Proceedings of the 18th European Conference on Computer Systems, EuroSys 2023 |
Pages | 416-432 |
Conference Date | 2023/05/08-2023/05/12 |
Conference Place | Rome |
Publisher | Association for Computing Machinery, Inc |
Abstract | To fully utilize computing resources, cloud providers such as Google and Alibaba choose to co-locate online services with batch processing applications in their data centers. By implementing unified resource management policies, different types of complex computing jobs request resources in a consistent way, which can help data centers achieve global optimal scheduling and provide computing power with higher quality. To understand this new scheduling paradigm, in this paper, we first present an in-depth study of Alibaba’s unified scheduling workloads. Our study focuses on the characterization of resource utilization, the application running performance, and scheduling scalability. We observe that although computing resources are significantly over-committed under unified scheduling, the resource utilization in Alibaba data centers is still low. In addition, existing resource usage predictors tend to make severe overestimations. At the same time, tasks within the same application behave fairly consistently, and the running performance of tasks can be well-profiled with respect to resource contention on the corresponding physical host. Based on these observations, in this paper, we design Optum, a unified data center scheduler for improving the overall resource utilization while ensuring good performance for each application. Optum formulates an optimization problem to schedule unified task requests, aiming to balance the trade-off between utilization and resource contention. Optum also implements efficient heuristics to solve the optimization problem in a scalable manner. Large-scale experiments demonstrate that Optum can save up to 15% of resources without performance degradation compared to state-of-the-art unified scheduling schemes. |
Keyword | Cloud Computing Resource Over-commitment Unified Scheduling |
DOI | 10.1145/3552326.3587437 |
URL | View the original |
Indexed By | CPCI-S |
Funding Project | Software-defined Methods and Key Technologies for Intelligent Control of Cloud Data Centres ; Efficient Integration and Dynamic Cognitive Technology and Platform for Urban Public Services |
Language | 英語English |
WOS Research Area | Computer Science |
WOS Subject | Computer Science, Hardware & Architecture ; Computer Science, Information Systems ; Computer Science, Software Engineering ; Computer Science, Theory & Methods |
WOS ID | WOS:001062106700026 |
Scopus ID | 2-s2.0-85160209377 |
Fulltext Access | |
Citation statistics | |
Document Type | Conference paper |
Collection | DEPARTMENT OF COMPUTER AND INFORMATION SCIENCE Faculty of Science and Technology |
Affiliation | 1.Shenzhen Institute of Advanced Technology, CAS Univ. of CAS, Univ. of Macau Macau SAR, China 2.University of Macau Macau SAR, China 3.Shenzhen Institute of Advanced Technology, CAS, Shenzhen, China 4.Alibaba Group, Hangzhou, China |
First Author Affilication | University of Macau |
Recommended Citation GB/T 7714 | Lu, Chengzhi,Xu, Huanle,Ye, Kejiang,et al. Understanding and Optimizing Workloads for Unified Resource Management in Large Cloud Platforms[C]:Association for Computing Machinery, Inc, 2023, 416-432. |
APA | Lu, Chengzhi., Xu, Huanle., Ye, Kejiang., Xu, Guoyao., Zhang, Liping., Yang, Guodong., & Xu, Chengzhong (2023). Understanding and Optimizing Workloads for Unified Resource Management in Large Cloud Platforms. Proceedings of the 18th European Conference on Computer Systems, EuroSys 2023, 416-432. |
Files in This Item: | There are no files associated with this item. |
Items in the repository are protected by copyright, with all rights reserved, unless otherwise indicated.
Edit Comment