Residential College | false |
Status | 已發表Published |
Parallel Self-Attention and Spatial-Attention Fusion for Human Pose Estimation and Running Movement Recognition | |
Wu,Qingtian1; Zhang,Yu2; Zhang,Liming1; Yu,Haoyong3 | |
2023 | |
Source Publication | IEEE Transactions on Cognitive and Developmental Systems |
ISSN | 2379-8920 |
Volume | 16Issue:1Pages:358-368 |
Abstract | Human pose estimation (HPE) is a fundamental yet promising visual recognition problem. Existing popular methods (e.g., Hourglass and its variants) either attempt to directly add local features element-wisely, or (e.g., vision transformers) try to learn the global relationships among different human parts. However, it remains an open problem to effectively integrate the local-global representations for accurate HPE. In this work, we design four feature fusion strategies on the hierarchical ResNet structure, including direct channel concatenation, element-wise addition and two parallel structures. Both two parallel structures adopt the naive self-attention encoder to model global dependencies. The difference between them is that one adopts the original ResNet BottleNeck while the other employs a spatial-attention module (named SSF) to learn the local patterns. Experiments on COCO Keypoint 2017 show that our SSF for HPE (named SSPose) achieves the best average precision with acceptable computational cost among the compared state-of-the-art methods. In addition, we build a lightweight running dataset to verify the effectiveness of SSPose. Based solely on the keypoints estimated by our SSPose, we propose a regression model to identify valid running movements without training any other classifiers. Our source codes and running dataset are publicly available. |
Keyword | Convolutional Neural Networks Feature Extraction Feature Fusion Human Pose Estimation Pose Estimation Running Recognition Self-attention Semantics Spatial-attention Task Analysis Transformers Visualization |
DOI | 10.1109/TCDS.2023.3275652 |
URL | View the original |
Indexed By | SCIE |
Language | 英語English |
WOS Research Area | Computer Science ; Robotics ; Neurosciences & Neurology |
WOS Subject | Computer Science, Artificial Intelligence ; Robotics ; Neurosciences |
WOS ID | WOS:001167556100025 |
Publisher | Institute of Electrical and Electronics Engineers Inc. |
Scopus ID | 2-s2.0-85162876913 |
Fulltext Access | |
Citation statistics | |
Document Type | Journal article |
Collection | DEPARTMENT OF COMPUTER AND INFORMATION SCIENCE |
Corresponding Author | Zhang,Liming |
Affiliation | 1.Department of Computer and Information Science, Faculty of Sciences and Technology, University of Macau, China 2.Faculty of Sciences and Technology, University of Macau, Macau, China 3.Department of Biomedical Engineering, National University of Singapore, Singapore, Singapore |
First Author Affilication | University of Macau |
Corresponding Author Affilication | University of Macau |
Recommended Citation GB/T 7714 | Wu,Qingtian,Zhang,Yu,Zhang,Liming,et al. Parallel Self-Attention and Spatial-Attention Fusion for Human Pose Estimation and Running Movement Recognition[J]. IEEE Transactions on Cognitive and Developmental Systems, 2023, 16(1), 358-368. |
APA | Wu,Qingtian., Zhang,Yu., Zhang,Liming., & Yu,Haoyong (2023). Parallel Self-Attention and Spatial-Attention Fusion for Human Pose Estimation and Running Movement Recognition. IEEE Transactions on Cognitive and Developmental Systems, 16(1), 358-368. |
MLA | Wu,Qingtian,et al."Parallel Self-Attention and Spatial-Attention Fusion for Human Pose Estimation and Running Movement Recognition".IEEE Transactions on Cognitive and Developmental Systems 16.1(2023):358-368. |
Files in This Item: | There are no files associated with this item. |
Items in the repository are protected by copyright, with all rights reserved, unless otherwise indicated.
Edit Comment