Residential Collegefalse
Status已發表Published
Parallel Self-Attention and Spatial-Attention Fusion for Human Pose Estimation and Running Movement Recognition
Wu,Qingtian1; Zhang,Yu2; Zhang,Liming1; Yu,Haoyong3
2023
Source PublicationIEEE Transactions on Cognitive and Developmental Systems
ISSN2379-8920
Volume16Issue:1Pages:358-368
Abstract

Human pose estimation (HPE) is a fundamental yet promising visual recognition problem. Existing popular methods (e.g., Hourglass and its variants) either attempt to directly add local features element-wisely, or (e.g., vision transformers) try to learn the global relationships among different human parts. However, it remains an open problem to effectively integrate the local-global representations for accurate HPE. In this work, we design four feature fusion strategies on the hierarchical ResNet structure, including direct channel concatenation, element-wise addition and two parallel structures. Both two parallel structures adopt the naive self-attention encoder to model global dependencies. The difference between them is that one adopts the original ResNet BottleNeck while the other employs a spatial-attention module (named SSF) to learn the local patterns. Experiments on COCO Keypoint 2017 show that our SSF for HPE (named SSPose) achieves the best average precision with acceptable computational cost among the compared state-of-the-art methods. In addition, we build a lightweight running dataset to verify the effectiveness of SSPose. Based solely on the keypoints estimated by our SSPose, we propose a regression model to identify valid running movements without training any other classifiers. Our source codes and running dataset are publicly available.

KeywordConvolutional Neural Networks Feature Extraction Feature Fusion Human Pose Estimation Pose Estimation Running Recognition Self-attention Semantics Spatial-attention Task Analysis Transformers Visualization
DOI10.1109/TCDS.2023.3275652
URLView the original
Indexed BySCIE
Language英語English
WOS Research AreaComputer Science ; Robotics ; Neurosciences & Neurology
WOS SubjectComputer Science, Artificial Intelligence ; Robotics ; Neurosciences
WOS IDWOS:001167556100025
PublisherInstitute of Electrical and Electronics Engineers Inc.
Scopus ID2-s2.0-85162876913
Fulltext Access
Citation statistics
Document TypeJournal article
CollectionDEPARTMENT OF COMPUTER AND INFORMATION SCIENCE
Corresponding AuthorZhang,Liming
Affiliation1.Department of Computer and Information Science, Faculty of Sciences and Technology, University of Macau, China
2.Faculty of Sciences and Technology, University of Macau, Macau, China
3.Department of Biomedical Engineering, National University of Singapore, Singapore, Singapore
First Author AffilicationUniversity of Macau
Corresponding Author AffilicationUniversity of Macau
Recommended Citation
GB/T 7714
Wu,Qingtian,Zhang,Yu,Zhang,Liming,et al. Parallel Self-Attention and Spatial-Attention Fusion for Human Pose Estimation and Running Movement Recognition[J]. IEEE Transactions on Cognitive and Developmental Systems, 2023, 16(1), 358-368.
APA Wu,Qingtian., Zhang,Yu., Zhang,Liming., & Yu,Haoyong (2023). Parallel Self-Attention and Spatial-Attention Fusion for Human Pose Estimation and Running Movement Recognition. IEEE Transactions on Cognitive and Developmental Systems, 16(1), 358-368.
MLA Wu,Qingtian,et al."Parallel Self-Attention and Spatial-Attention Fusion for Human Pose Estimation and Running Movement Recognition".IEEE Transactions on Cognitive and Developmental Systems 16.1(2023):358-368.
Files in This Item:
There are no files associated with this item.
Related Services
Recommend this item
Bookmark
Usage statistics
Export to Endnote
Google Scholar
Similar articles in Google Scholar
[Wu,Qingtian]'s Articles
[Zhang,Yu]'s Articles
[Zhang,Liming]'s Articles
Baidu academic
Similar articles in Baidu academic
[Wu,Qingtian]'s Articles
[Zhang,Yu]'s Articles
[Zhang,Liming]'s Articles
Bing Scholar
Similar articles in Bing Scholar
[Wu,Qingtian]'s Articles
[Zhang,Yu]'s Articles
[Zhang,Liming]'s Articles
Terms of Use
No data!
Social Bookmark/Share
All comments (0)
No comment.
 

Items in the repository are protected by copyright, with all rights reserved, unless otherwise indicated.