Residential College | false |
Status | 已發表Published |
Adaptive Pitfall: Exploring the Effectiveness of Adaptation in Skeleton-based Action Recognition | |
Miao, Qiguang1; Xin, Wentian1; Liu, Ruyi1![]() ![]() | |
2024-12 | |
Source Publication | IEEE Transactions on Multimedia
![]() |
ISSN | 1520-9210 |
Volume | 27Pages:56-71 |
Abstract | Graph convolution networks (GCNs) have achieved remarkable performance in skeleton-based action recognition by exploiting the adjacency topology of body representation. However, the adaptive strategy adopted by the previous methods to construct the adjacency matrix is not balanced between the performance and the computational cost. We assume this concept of Adaptive Trap, which can be replaced by multiple autonomous submodules, thereby simultaneously enhancing the dynamic joint representation and effectively reducing network resources. To effectuate the substitution of the adaptive model, we unveil two distinct strategies, both yielding comparable effects: (1) Optimization. Individuality and Commonality GCNs (IC-GCNs) is proposed to specifically optimize the construction method of the associativity adjacency matrix for adaptive processing. The uniqueness and co-occurrence between different joint points and frames in the skeleton topology are effectively captured through methodologies like preferential fusion of physical information, extreme compression of multi-dimensional channels, and simplification of self-attention mechanism. (2) Replacement. AutoLearning GCNs (AL-GCNs) is proposed to boldly remove popular adaptive modules and cleverly utilize human key points as motion compensation to provide dynamic correlation support. AL-GCNs construct a fully learnable group adjacency matrix in both spatial and temporal dimensions, resulting in an elegant and efficient GCN-based model. In addition, three effective tricks for skeletonbased action recognition (Skip-Block, Bayesian Weight Selection Algorithm, and Simplified Dimensional Attention) are exposed and analyzed in this paper. Finally, we employ the variable channel and grouping method to explore the hardware resource bound of the two proposed models. IC-GCN and AL-GCN exhibit impressive performance across NTU-RGB+D 60, NTU-RGB+D 120, NW-UCLA, and UAV-Human datasets, with an exceptional parameter-cost ratio. Code will be available on Adaptive-Pitfall. |
Keyword | Skeleton-based Action Recognition Graph Convolutional Network Self-attention Method Lightweight Network |
DOI | 10.1109/TMM.2024.3521774 |
URL | View the original |
Language | 英語English |
Scopus ID | 2-s2.0-85213484074 |
Fulltext Access | |
Citation statistics | |
Document Type | Journal article |
Collection | DEPARTMENT OF COMPUTER AND INFORMATION SCIENCE |
Corresponding Author | Liu, Ruyi |
Affiliation | 1.Xidian University, School of Computer Science and Technology, Xi'an, Shanxi, 710071, China 2.Xi'an University of Technology, School of Computer Science and Engineering, Xi'an, Shanxi, 710048, China 3.University of Macau, Avenida da Universidade, Faculty of Science and Technology, Department of Computer and Information Science, Taipa, Macao |
Recommended Citation GB/T 7714 | Miao, Qiguang,Xin, Wentian,Liu, Ruyi,et al. Adaptive Pitfall: Exploring the Effectiveness of Adaptation in Skeleton-based Action Recognition[J]. IEEE Transactions on Multimedia, 2024, 27, 56-71. |
APA | Miao, Qiguang., Xin, Wentian., Liu, Ruyi., Liu, Yi., Wu, Mengyao., Shi, Cheng., & Pun, Chi Man (2024). Adaptive Pitfall: Exploring the Effectiveness of Adaptation in Skeleton-based Action Recognition. IEEE Transactions on Multimedia, 27, 56-71. |
MLA | Miao, Qiguang,et al."Adaptive Pitfall: Exploring the Effectiveness of Adaptation in Skeleton-based Action Recognition".IEEE Transactions on Multimedia 27(2024):56-71. |
Files in This Item: | There are no files associated with this item. |
Items in the repository are protected by copyright, with all rights reserved, unless otherwise indicated.
Edit Comment