Residential College | false |
Status | 已發表Published |
Optimizing job scheduling by using broad learning to predict execution times on HPC clusters | |
Hou, Zhengxiong1; Shen, Hong2; Feng, Qiying3; Lv, Zhiqi4; Jin, Junwei5; Zhou, Xingshe1; Gu, Jianhua1 | |
2024-08 | |
Source Publication | CCF Transactions on High Performance Computing |
ISSN | 2524-4922 |
Volume | 6Issue:4Pages:365-377 |
Abstract | Small and middle size high-performance computing clusters are very popular for various applications. How to utilize the accumulated log data generated in the past to optimize job scheduling using machine learning techniques is an interesting problem. Most of the current work use the common machine learning algorithms, such as the multivariate linear regression and polynomial model, to predict job runtime and optimize job scheduling. They either ignore the interference among job features or require a high time overhead for improving the prediction accuracy. In this paper, we propose to implement and improve broad learning algorithm for predicting the execution times of new coming jobs more accurately and efficiently. The experimental results showed that the proposed method can obtain high prediction accuracy with a negligible time overhead. And the predicted job execution time can help improve the efficiency of job scheduling and HPC systems. |
Keyword | Parallel Systems Hpc Clusters Job Scheduling Runtime Prediction Broad Learning |
DOI | 10.1007/s42514-023-00137-z |
URL | View the original |
Indexed By | ESCI |
Language | 英語English |
WOS Research Area | Computer Science |
WOS Subject | Computer Science, Hardware & Architecture ; Computer Science, Information Systems ; Computer Science, Interdisciplinary Applications ; Computer Science, Theory & Methods |
WOS ID | WOS:000937180600001 |
Publisher | SPRINGERNATURE, CAMPUS, 4 CRINAN ST, LONDON N1 9XW, ENGLAND |
Scopus ID | 2-s2.0-85148577003 |
Fulltext Access | |
Citation statistics | |
Document Type | Journal article |
Collection | Faculty of Science and Technology |
Corresponding Author | Hou, Zhengxiong |
Affiliation | 1.Center for High-Performance Computing, School of Computer Science, Northwestern Polytechnical University, Xi’an, Shaanxi, 710072, China 2.School of Computer Science, Sun Yat-Sen University, Guangzhou, China 3.Faculty of Science and Technology, University of Macau, Macao 4.School of Computer Science, Northwestern Polytechnical University, Xi’an, China 5.School of Information Science and Engineering, He-Nan University of Technology, Zhengzhou, China |
Recommended Citation GB/T 7714 | Hou, Zhengxiong,Shen, Hong,Feng, Qiying,et al. Optimizing job scheduling by using broad learning to predict execution times on HPC clusters[J]. CCF Transactions on High Performance Computing, 2024, 6(4), 365-377. |
APA | Hou, Zhengxiong., Shen, Hong., Feng, Qiying., Lv, Zhiqi., Jin, Junwei., Zhou, Xingshe., & Gu, Jianhua (2024). Optimizing job scheduling by using broad learning to predict execution times on HPC clusters. CCF Transactions on High Performance Computing, 6(4), 365-377. |
MLA | Hou, Zhengxiong,et al."Optimizing job scheduling by using broad learning to predict execution times on HPC clusters".CCF Transactions on High Performance Computing 6.4(2024):365-377. |
Files in This Item: | There are no files associated with this item. |
Items in the repository are protected by copyright, with all rights reserved, unless otherwise indicated.
Edit Comment