Residential Collegefalse
Status已發表Published
Adaptive Channel Sparsity for Federated Learning under System Heterogeneity
Liao, Dongping1; Gao, Xitong2; Zhao, Yiren3; Xu, Chengzhong1
2023
Conference NameIEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
Source PublicationProceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition
Volume2023-June
Pages20432-20441
Conference DateJUN 17-24, 2023
Conference PlaceVancouver
PublisherIEEE COMPUTER SOC10662 LOS VAQUEROS CIRCLE, PO BOX 3014, LOS ALAMITOS, CA 90720-1264 USA
Abstract

Owing to the non-i.i.d. nature of client data, channel neurons in federated-learned models may specialize to distinct features for different clients. Yet, existing channel-sparse federated learning (FL) algorithms prescribe fixed sparsity strategies for client models, and may thus prevent clients from training channel neurons collaboratively. To minimize the impact of sparsity on FL convergence, we propose Flado to improve the alignment of client model update trajectories by tailoring the sparsities of individual neurons in each client. Empirical results show that while other sparse methods are surprisingly impactful to convergence, Flado can not only attain the highest task accuracies with unlimited budget across a range of datasets, but also significantly reduce the amount of floating-point operations (FLOPs) required for training more than by 10× under the same communications budget, and push the Pareto frontier of communication/computation trade-off notably further than competing FL algorithms.

KeywordEfficient And Scalable Vision
DOI10.1109/CVPR52729.2023.01957
URLView the original
Indexed ByCPCI-S
Language英語English
WOS Research AreaComputer Science
WOS SubjectComputer Science, Artificial Intelligence
WOS IDWOS:001062531304073
Scopus ID2-s2.0-85172429032
Fulltext Access
Citation statistics
Document TypeConference paper
CollectionTHE STATE KEY LABORATORY OF INTERNET OF THINGS FOR SMART CITY (UNIVERSITY OF MACAU)
Faculty of Science and Technology
DEPARTMENT OF COMPUTER AND INFORMATION SCIENCE
Corresponding AuthorXu, Chengzhong
Affiliation1.University of Macau, State Key Lab of IoTSC, Cis Dept, Macao
2.Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences, Shenzhen, China
3.Imperial College London, London, United Kingdom
First Author AffilicationUniversity of Macau
Corresponding Author AffilicationUniversity of Macau
Recommended Citation
GB/T 7714
Liao, Dongping,Gao, Xitong,Zhao, Yiren,et al. Adaptive Channel Sparsity for Federated Learning under System Heterogeneity[C]:IEEE COMPUTER SOC10662 LOS VAQUEROS CIRCLE, PO BOX 3014, LOS ALAMITOS, CA 90720-1264 USA, 2023, 20432-20441.
APA Liao, Dongping., Gao, Xitong., Zhao, Yiren., & Xu, Chengzhong (2023). Adaptive Channel Sparsity for Federated Learning under System Heterogeneity. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2023-June, 20432-20441.
Files in This Item:
There are no files associated with this item.
Related Services
Recommend this item
Bookmark
Usage statistics
Export to Endnote
Google Scholar
Similar articles in Google Scholar
[Liao, Dongping]'s Articles
[Gao, Xitong]'s Articles
[Zhao, Yiren]'s Articles
Baidu academic
Similar articles in Baidu academic
[Liao, Dongping]'s Articles
[Gao, Xitong]'s Articles
[Zhao, Yiren]'s Articles
Bing Scholar
Similar articles in Bing Scholar
[Liao, Dongping]'s Articles
[Gao, Xitong]'s Articles
[Zhao, Yiren]'s Articles
Terms of Use
No data!
Social Bookmark/Share
All comments (0)
No comment.
 

Items in the repository are protected by copyright, with all rights reserved, unless otherwise indicated.