UM  > Faculty of Science and Technology
Residential Collegefalse
Status已發表Published
Optimizing job scheduling by using broad learning to predict execution times on HPC clusters
Hou, Zhengxiong1; Shen, Hong2; Feng, Qiying3; Lv, Zhiqi4; Jin, Junwei5; Zhou, Xingshe1; Gu, Jianhua1
2024-08
Source PublicationCCF Transactions on High Performance Computing
ISSN2524-4922
Volume6Issue:4Pages:365-377
Abstract

Small and middle size high-performance computing clusters are very popular for various applications. How to utilize the accumulated log data generated in the past to optimize job scheduling using machine learning techniques is an interesting problem. Most of the current work use the common machine learning algorithms, such as the multivariate linear regression and polynomial model, to predict job runtime and optimize job scheduling. They either ignore the interference among job features or require a high time overhead for improving the prediction accuracy. In this paper, we propose to implement and improve broad learning algorithm for predicting the execution times of new coming jobs more accurately and efficiently. The experimental results showed that the proposed method can obtain high prediction accuracy with a negligible time overhead. And the predicted job execution time can help improve the efficiency of job scheduling and HPC systems.

KeywordParallel Systems Hpc Clusters Job Scheduling Runtime Prediction Broad Learning
DOI10.1007/s42514-023-00137-z
URLView the original
Indexed ByESCI
Language英語English
WOS Research AreaComputer Science
WOS SubjectComputer Science, Hardware & Architecture ; Computer Science, Information Systems ; Computer Science, Interdisciplinary Applications ; Computer Science, Theory & Methods
WOS IDWOS:000937180600001
PublisherSPRINGERNATURE, CAMPUS, 4 CRINAN ST, LONDON N1 9XW, ENGLAND
Scopus ID2-s2.0-85148577003
Fulltext Access
Citation statistics
Document TypeJournal article
CollectionFaculty of Science and Technology
Corresponding AuthorHou, Zhengxiong
Affiliation1.Center for High-Performance Computing, School of Computer Science, Northwestern Polytechnical University, Xi’an, Shaanxi, 710072, China
2.School of Computer Science, Sun Yat-Sen University, Guangzhou, China
3.Faculty of Science and Technology, University of Macau, Macao
4.School of Computer Science, Northwestern Polytechnical University, Xi’an, China
5.School of Information Science and Engineering, He-Nan University of Technology, Zhengzhou, China
Recommended Citation
GB/T 7714
Hou, Zhengxiong,Shen, Hong,Feng, Qiying,et al. Optimizing job scheduling by using broad learning to predict execution times on HPC clusters[J]. CCF Transactions on High Performance Computing, 2024, 6(4), 365-377.
APA Hou, Zhengxiong., Shen, Hong., Feng, Qiying., Lv, Zhiqi., Jin, Junwei., Zhou, Xingshe., & Gu, Jianhua (2024). Optimizing job scheduling by using broad learning to predict execution times on HPC clusters. CCF Transactions on High Performance Computing, 6(4), 365-377.
MLA Hou, Zhengxiong,et al."Optimizing job scheduling by using broad learning to predict execution times on HPC clusters".CCF Transactions on High Performance Computing 6.4(2024):365-377.
Files in This Item:
There are no files associated with this item.
Related Services
Recommend this item
Bookmark
Usage statistics
Export to Endnote
Google Scholar
Similar articles in Google Scholar
[Hou, Zhengxiong]'s Articles
[Shen, Hong]'s Articles
[Feng, Qiying]'s Articles
Baidu academic
Similar articles in Baidu academic
[Hou, Zhengxiong]'s Articles
[Shen, Hong]'s Articles
[Feng, Qiying]'s Articles
Bing Scholar
Similar articles in Bing Scholar
[Hou, Zhengxiong]'s Articles
[Shen, Hong]'s Articles
[Feng, Qiying]'s Articles
Terms of Use
No data!
Social Bookmark/Share
All comments (0)
No comment.
 

Items in the repository are protected by copyright, with all rights reserved, unless otherwise indicated.