Residential College | false |
Status | 已發表Published |
EINS: Edge-Cloud Deep Model Inference with Network-Efficiency Schedule in Serverless | |
Peng, Shijie1,2; Lin, Yanying1,2; Chen, Wenyan1,3; Tang, Yingfei1; Duan, Xu1,2; Ye, Kejiang1 | |
2024-07 | |
Conference Name | 2024 27th International Conference on Computer Supported Cooperative Work in Design (CSCWD) |
Source Publication | Proceedings of the 2024 27th International Conference on Computer Supported Cooperative Work in Design, CSCWD 2024 |
Pages | 1376-1381 |
Conference Date | 08-10 May 2024 |
Conference Place | Tianjin, China |
Country | China |
Publisher | IEEE |
Abstract | Model inference in edge is often regarded as an effective method to alleviate high latency and enhance data privacy in edge-cloud collaborative computing environment. In this paper, we demonstrate that optimizing network communication in edge-cloud environment with limited bandwidth can enhance model inference performance. We first analyze network bottlenecks and the characteristics in model inference, then design a serverless inference system -EINS, to support collaborative optimization of network transmission and inference performance in edge-cloud environment. This system identifies concurrent network communication bottlenecks in multi-model deployment, dynamically scales capacity, and optimizes placement strategies and model transfer sequences. Real-world workload evaluation reveal that EINS can achieve a 5.7x throughput improvement and an average reduction of 62% latency in model instance startup. |
Keyword | Edge-cloud Collaborative Network-efficiency Serverless Inference |
DOI | 10.1109/CSCWD61410.2024.10580052 |
URL | View the original |
Language | 英語English |
Scopus ID | 2-s2.0-85199091900 |
Fulltext Access | |
Citation statistics | |
Document Type | Conference paper |
Collection | Faculty of Science and Technology |
Corresponding Author | Ye, Kejiang |
Affiliation | 1.Chinese Academy of Sciences, Shenzhen Institute of Advanced Technology, China 2.University of Chinese Academy of Sciences, China 3.University of Macau, Macao |
Recommended Citation GB/T 7714 | Peng, Shijie,Lin, Yanying,Chen, Wenyan,et al. EINS: Edge-Cloud Deep Model Inference with Network-Efficiency Schedule in Serverless[C]:IEEE, 2024, 1376-1381. |
APA | Peng, Shijie., Lin, Yanying., Chen, Wenyan., Tang, Yingfei., Duan, Xu., & Ye, Kejiang (2024). EINS: Edge-Cloud Deep Model Inference with Network-Efficiency Schedule in Serverless. Proceedings of the 2024 27th International Conference on Computer Supported Cooperative Work in Design, CSCWD 2024, 1376-1381. |
Files in This Item: | There are no files associated with this item. |
Items in the repository are protected by copyright, with all rights reserved, unless otherwise indicated.
Edit Comment