Residential Collegefalse
Status已發表Published
Understanding the Workload Characteristics in Alibaba: A View from Directed Acyclic Graph Analysis
Lu,Chengzhi1,2; Chen,Wenyan1; Ye,Kejiang1; Xu,Cheng Zhong3
2020-07-01
Conference Name2020 International Conference on High Performance Big Data and Intelligent Systems (HPBD&IS)
Source Publication2020 International Conference on High Performance Big Data and Intelligent Systems (HPBD and IS 2020)
Conference Date2020-05-23
Conference PlaceShenzhen, China
PublisherInstitute of Electrical and Electronics Engineers Inc.
Abstract

Cloud computing technology is widely used in data centers due to the benefits such as high scalability, high availability, on-demand provision, and so on. Understanding the characteristics of complex workloads is crucial for job scheduling, capacity planning, and performance optimization, which provide guarantees for the normal operation of cloud-native applications. Co-location of online services and batch jobs is a common way to improve resource utilization in today's cloud data centers. It is very important for the resource scheduler to adapt to the characteristics of complex and varied workloads. In this paper, we perform an in-depth analysis on the latest released trace dataset by Alibaba in December 2018, consists of 4195049 batch jobs and 71476 containers co-locating on about 4000 machines. Different from Alibaba trace 2017, the information of directed acyclic graph (DAG) is added in the trace, which explains the relationship between the jobs and tasks. We first show the basic statics of the cluster. Then, we conducted an in-depth analysis of the workload types and DAG information. The results reveal several new insights: i) The resource utilization in the cluster shows an obvious periodic phenomenon (day and week). ii) There is a resource (e.g. CPU, memory usage) overprovisioning phenomenon in the cluster. iii) The workloads' characteristics of different workload types are very different. iv) The distribution of instance startup time and termination time is unbalanced. v) 90% of the job's DAG length does not exceed 30 while the maximum length is 54.

KeywordCloud Computing Workload Characterization Dependency Graph Analysis Workload Type
DOI10.1109/HPBDIS49115.2020.9130578
URLView the original
Language英語English
Scopus ID2-s2.0-85091968279
Fulltext Access
Citation statistics
Document TypeConference paper
CollectionTHE STATE KEY LABORATORY OF INTERNET OF THINGS FOR SMART CITY (UNIVERSITY OF MACAU)
Faculty of Science and Technology
Corresponding AuthorYe,Kejiang
Affiliation1.Shenzhen Institutes of Advanced Technology,Chinese Academy of Sciences,China
2.University of Chinese Academy of Sciences,China
3.State Key Lab of IoTSC,Faculty of Science and Technology,University of Macau,Macao
Recommended Citation
GB/T 7714
Lu,Chengzhi,Chen,Wenyan,Ye,Kejiang,et al. Understanding the Workload Characteristics in Alibaba: A View from Directed Acyclic Graph Analysis[C]:Institute of Electrical and Electronics Engineers Inc., 2020.
APA Lu,Chengzhi., Chen,Wenyan., Ye,Kejiang., & Xu,Cheng Zhong (2020). Understanding the Workload Characteristics in Alibaba: A View from Directed Acyclic Graph Analysis. 2020 International Conference on High Performance Big Data and Intelligent Systems (HPBD and IS 2020).
Files in This Item:
There are no files associated with this item.
Related Services
Recommend this item
Bookmark
Usage statistics
Export to Endnote
Google Scholar
Similar articles in Google Scholar
[Lu,Chengzhi]'s Articles
[Chen,Wenyan]'s Articles
[Ye,Kejiang]'s Articles
Baidu academic
Similar articles in Baidu academic
[Lu,Chengzhi]'s Articles
[Chen,Wenyan]'s Articles
[Ye,Kejiang]'s Articles
Bing Scholar
Similar articles in Bing Scholar
[Lu,Chengzhi]'s Articles
[Chen,Wenyan]'s Articles
[Ye,Kejiang]'s Articles
Terms of Use
No data!
Social Bookmark/Share
All comments (0)
No comment.
 

Items in the repository are protected by copyright, with all rights reserved, unless otherwise indicated.