UM  > Faculty of Science and Technology
Residential Collegefalse
Status已發表Published
EINS: Edge-Cloud Deep Model Inference with Network-Efficiency Schedule in Serverless
Peng, Shijie1,2; Lin, Yanying1,2; Chen, Wenyan1,3; Tang, Yingfei1; Duan, Xu1,2; Ye, Kejiang1
2024-07
Conference Name2024 27th International Conference on Computer Supported Cooperative Work in Design (CSCWD)
Source PublicationProceedings of the 2024 27th International Conference on Computer Supported Cooperative Work in Design, CSCWD 2024
Pages1376-1381
Conference Date08-10 May 2024
Conference PlaceTianjin, China
CountryChina
PublisherIEEE
Abstract

Model inference in edge is often regarded as an effective method to alleviate high latency and enhance data privacy in edge-cloud collaborative computing environment. In this paper, we demonstrate that optimizing network communication in edge-cloud environment with limited bandwidth can enhance model inference performance. We first analyze network bottlenecks and the characteristics in model inference, then design a serverless inference system -EINS, to support collaborative optimization of network transmission and inference performance in edge-cloud environment. This system identifies concurrent network communication bottlenecks in multi-model deployment, dynamically scales capacity, and optimizes placement strategies and model transfer sequences. Real-world workload evaluation reveal that EINS can achieve a 5.7x throughput improvement and an average reduction of 62% latency in model instance startup.

KeywordEdge-cloud Collaborative Network-efficiency Serverless Inference
DOI10.1109/CSCWD61410.2024.10580052
URLView the original
Language英語English
Scopus ID2-s2.0-85199091900
Fulltext Access
Citation statistics
Document TypeConference paper
CollectionFaculty of Science and Technology
Corresponding AuthorYe, Kejiang
Affiliation1.Chinese Academy of Sciences, Shenzhen Institute of Advanced Technology, China
2.University of Chinese Academy of Sciences, China
3.University of Macau, Macao
Recommended Citation
GB/T 7714
Peng, Shijie,Lin, Yanying,Chen, Wenyan,et al. EINS: Edge-Cloud Deep Model Inference with Network-Efficiency Schedule in Serverless[C]:IEEE, 2024, 1376-1381.
APA Peng, Shijie., Lin, Yanying., Chen, Wenyan., Tang, Yingfei., Duan, Xu., & Ye, Kejiang (2024). EINS: Edge-Cloud Deep Model Inference with Network-Efficiency Schedule in Serverless. Proceedings of the 2024 27th International Conference on Computer Supported Cooperative Work in Design, CSCWD 2024, 1376-1381.
Files in This Item:
There are no files associated with this item.
Related Services
Recommend this item
Bookmark
Usage statistics
Export to Endnote
Google Scholar
Similar articles in Google Scholar
[Peng, Shijie]'s Articles
[Lin, Yanying]'s Articles
[Chen, Wenyan]'s Articles
Baidu academic
Similar articles in Baidu academic
[Peng, Shijie]'s Articles
[Lin, Yanying]'s Articles
[Chen, Wenyan]'s Articles
Bing Scholar
Similar articles in Bing Scholar
[Peng, Shijie]'s Articles
[Lin, Yanying]'s Articles
[Chen, Wenyan]'s Articles
Terms of Use
No data!
Social Bookmark/Share
All comments (0)
No comment.
 

Items in the repository are protected by copyright, with all rights reserved, unless otherwise indicated.