Residential College | false |
Status | 已發表Published |
Spatio-temporal compression for semi-supervised video object segmentation | |
Ji, Chuanjun1,2; Chen, Yadang1,2; Yang, Zhi Xin3; Wu, Enhua4 | |
2022-08-13 | |
Source Publication | Visual Computer |
ISSN | 0178-2789 |
Volume | 39Issue:10Pages:4929-4942 |
Abstract | In this paper, we explore the spatial–temporal redundancy in video object segmentation (VOS) under semi-supervised context with the purpose to improve the computational efficiency. Recently, memory-based methods have attracted great attention for their excellent performance. These methods involve first constructing an external memory to store the target object information in the history frames and then selecting the information that is beneficial for modeling the target object by memory reading. However, such methods are inefficient and unable to achieve both high accuracy and high efficiency, due to the large amount of redundant information in memory. Moreover, they periodically sample historical frames and add them to memory; this operation may lose important information from dynamic frames with incremental object changing or aggravate temporal redundancy from static frames with no object changing. To address these problems, we propose an efficient semi-supervised VOS approach via spatio-temporal compression (termed as STCVOS). Specifically, we first adopt a temporally varying sensor to adaptively filter static frames with no target objects evolutions and trigger memory to update with frames containing noticeable variations. Furthermore, we propose a spatially compressed memory to absorb features with varied pixels and remove outdated features, which considerably reduces information redundancy. More importantly, we introduce an efficient memory reader to perform memory reading with less footprint and computational overhead. Experimental results indicate that STCVOS performs well against state-of-the-art methods on the DAVIS 2017 and YouTube-VOS datasets, with a J& F overall score of 82.0% and 79.7%, respectively. Meanwhile, STCVOS achieves a high inference speed of approximately 30 FPS. |
Keyword | External Memory Memory Reading Spatial–temporal Redundancy Video Object Segmentation |
DOI | 10.1007/s00371-022-02638-4 |
URL | View the original |
Indexed By | SCIE |
Language | 英語English |
WOS Research Area | Computer Science |
WOS Subject | Computer Science, Software Engineering |
WOS ID | WOS:000840290600004 |
Publisher | SPRINGER |
Scopus ID | 2-s2.0-85136033935 |
Fulltext Access | |
Citation statistics | |
Document Type | Journal article |
Collection | THE STATE KEY LABORATORY OF INTERNET OF THINGS FOR SMART CITY (UNIVERSITY OF MACAU) Faculty of Science and Technology DEPARTMENT OF COMPUTER AND INFORMATION SCIENCE |
Corresponding Author | Chen, Yadang |
Affiliation | 1.School of Computer and Software, Nanjing University of Information Science and Technology, Nanjing, 210044, China 2.Engineering Research Center of Digital Forensics, Ministry of Education, Nanjing University of Information Science and Technology, Nanjing, 210044, China 3.The State Key Laboratory of Internet of Things for Smart City and Department of Electromechanical Engineering, University of Macau, 999078, Macao 4.State Key Laboratory of Computer Science, Institute of Software, University of Chinese Academy of Sciences, Beijing, 100190, China |
Recommended Citation GB/T 7714 | Ji, Chuanjun,Chen, Yadang,Yang, Zhi Xin,et al. Spatio-temporal compression for semi-supervised video object segmentation[J]. Visual Computer, 2022, 39(10), 4929-4942. |
APA | Ji, Chuanjun., Chen, Yadang., Yang, Zhi Xin., & Wu, Enhua (2022). Spatio-temporal compression for semi-supervised video object segmentation. Visual Computer, 39(10), 4929-4942. |
MLA | Ji, Chuanjun,et al."Spatio-temporal compression for semi-supervised video object segmentation".Visual Computer 39.10(2022):4929-4942. |
Files in This Item: | There are no files associated with this item. |
Items in the repository are protected by copyright, with all rights reserved, unless otherwise indicated.
Edit Comment