Residential Collegefalse
Status已發表Published
Adaptive Pitfall: Exploring the Effectiveness of Adaptation in Skeleton-based Action Recognition
Miao, Qiguang1; Xin, Wentian1; Liu, Ruyi1; Liu, Yi1; Wu, Mengyao1; Shi, Cheng2; Pun, Chi Man3
2024-12
Source PublicationIEEE Transactions on Multimedia
ISSN1520-9210
Volume27Pages:56-71
Abstract

Graph convolution networks (GCNs) have achieved remarkable performance in skeleton-based action recognition by exploiting the adjacency topology of body representation. However, the adaptive strategy adopted by the previous methods to construct the adjacency matrix is not balanced between the performance and the computational cost. We assume this concept of Adaptive Trap, which can be replaced by multiple autonomous submodules, thereby simultaneously enhancing the dynamic joint representation and effectively reducing network resources. To effectuate the substitution of the adaptive model, we unveil two distinct strategies, both yielding comparable effects: (1) Optimization. Individuality and Commonality GCNs (IC-GCNs) is proposed to specifically optimize the construction method of the associativity adjacency matrix for adaptive processing. The uniqueness and co-occurrence between different joint points and frames in the skeleton topology are effectively captured through methodologies like preferential fusion of physical information, extreme compression of multi-dimensional channels, and simplification of self-attention mechanism. (2) Replacement. AutoLearning GCNs (AL-GCNs) is proposed to boldly remove popular adaptive modules and cleverly utilize human key points as motion compensation to provide dynamic correlation support. AL-GCNs construct a fully learnable group adjacency matrix in both spatial and temporal dimensions, resulting in an elegant and efficient GCN-based model. In addition, three effective tricks for skeletonbased action recognition (Skip-Block, Bayesian Weight Selection Algorithm, and Simplified Dimensional Attention) are exposed and analyzed in this paper. Finally, we employ the variable channel and grouping method to explore the hardware resource bound of the two proposed models. IC-GCN and AL-GCN exhibit impressive performance across NTU-RGB+D 60, NTU-RGB+D 120, NW-UCLA, and UAV-Human datasets, with an exceptional parameter-cost ratio. Code will be available on Adaptive-Pitfall.

KeywordSkeleton-based Action Recognition Graph Convolutional Network Self-attention Method Lightweight Network
DOI10.1109/TMM.2024.3521774
URLView the original
Language英語English
Scopus ID2-s2.0-85213484074
Fulltext Access
Citation statistics
Document TypeJournal article
CollectionDEPARTMENT OF COMPUTER AND INFORMATION SCIENCE
Corresponding AuthorLiu, Ruyi
Affiliation1.Xidian University, School of Computer Science and Technology, Xi'an, Shanxi, 710071, China
2.Xi'an University of Technology, School of Computer Science and Engineering, Xi'an, Shanxi, 710048, China
3.University of Macau, Avenida da Universidade, Faculty of Science and Technology, Department of Computer and Information Science, Taipa, Macao
Recommended Citation
GB/T 7714
Miao, Qiguang,Xin, Wentian,Liu, Ruyi,et al. Adaptive Pitfall: Exploring the Effectiveness of Adaptation in Skeleton-based Action Recognition[J]. IEEE Transactions on Multimedia, 2024, 27, 56-71.
APA Miao, Qiguang., Xin, Wentian., Liu, Ruyi., Liu, Yi., Wu, Mengyao., Shi, Cheng., & Pun, Chi Man (2024). Adaptive Pitfall: Exploring the Effectiveness of Adaptation in Skeleton-based Action Recognition. IEEE Transactions on Multimedia, 27, 56-71.
MLA Miao, Qiguang,et al."Adaptive Pitfall: Exploring the Effectiveness of Adaptation in Skeleton-based Action Recognition".IEEE Transactions on Multimedia 27(2024):56-71.
Files in This Item:
There are no files associated with this item.
Related Services
Recommend this item
Bookmark
Usage statistics
Export to Endnote
Google Scholar
Similar articles in Google Scholar
[Miao, Qiguang]'s Articles
[Xin, Wentian]'s Articles
[Liu, Ruyi]'s Articles
Baidu academic
Similar articles in Baidu academic
[Miao, Qiguang]'s Articles
[Xin, Wentian]'s Articles
[Liu, Ruyi]'s Articles
Bing Scholar
Similar articles in Bing Scholar
[Miao, Qiguang]'s Articles
[Xin, Wentian]'s Articles
[Liu, Ruyi]'s Articles
Terms of Use
No data!
Social Bookmark/Share
All comments (0)
No comment.
 

Items in the repository are protected by copyright, with all rights reserved, unless otherwise indicated.