Residential Collegefalse
Status已發表Published
Spatio-temporal compression for semi-supervised video object segmentation
Ji, Chuanjun1,2; Chen, Yadang1,2; Yang, Zhi Xin3; Wu, Enhua4
2022-08-13
Source PublicationVisual Computer
ISSN0178-2789
Volume39Issue:10Pages:4929-4942
Abstract

In this paper, we explore the spatial–temporal redundancy in video object segmentation (VOS) under semi-supervised context with the purpose to improve the computational efficiency. Recently, memory-based methods have attracted great attention for their excellent performance. These methods involve first constructing an external memory to store the target object information in the history frames and then selecting the information that is beneficial for modeling the target object by memory reading. However, such methods are inefficient and unable to achieve both high accuracy and high efficiency, due to the large amount of redundant information in memory. Moreover, they periodically sample historical frames and add them to memory; this operation may lose important information from dynamic frames with incremental object changing or aggravate temporal redundancy from static frames with no object changing. To address these problems, we propose an efficient semi-supervised VOS approach via spatio-temporal compression (termed as STCVOS). Specifically, we first adopt a temporally varying sensor to adaptively filter static frames with no target objects evolutions and trigger memory to update with frames containing noticeable variations. Furthermore, we propose a spatially compressed memory to absorb features with varied pixels and remove outdated features, which considerably reduces information redundancy. More importantly, we introduce an efficient memory reader to perform memory reading with less footprint and computational overhead. Experimental results indicate that STCVOS performs well against state-of-the-art methods on the DAVIS 2017 and YouTube-VOS datasets, with a J& F overall score of 82.0% and 79.7%, respectively. Meanwhile, STCVOS achieves a high inference speed of approximately 30 FPS.

KeywordExternal Memory Memory Reading Spatial–temporal Redundancy Video Object Segmentation
DOI10.1007/s00371-022-02638-4
URLView the original
Indexed BySCIE
Language英語English
WOS Research AreaComputer Science
WOS SubjectComputer Science, Software Engineering
WOS IDWOS:000840290600004
PublisherSPRINGER
Scopus ID2-s2.0-85136033935
Fulltext Access
Citation statistics
Document TypeJournal article
CollectionTHE STATE KEY LABORATORY OF INTERNET OF THINGS FOR SMART CITY (UNIVERSITY OF MACAU)
Faculty of Science and Technology
DEPARTMENT OF COMPUTER AND INFORMATION SCIENCE
Corresponding AuthorChen, Yadang
Affiliation1.School of Computer and Software, Nanjing University of Information Science and Technology, Nanjing, 210044, China
2.Engineering Research Center of Digital Forensics, Ministry of Education, Nanjing University of Information Science and Technology, Nanjing, 210044, China
3.The State Key Laboratory of Internet of Things for Smart City and Department of Electromechanical Engineering, University of Macau, 999078, Macao
4.State Key Laboratory of Computer Science, Institute of Software, University of Chinese Academy of Sciences, Beijing, 100190, China
Recommended Citation
GB/T 7714
Ji, Chuanjun,Chen, Yadang,Yang, Zhi Xin,et al. Spatio-temporal compression for semi-supervised video object segmentation[J]. Visual Computer, 2022, 39(10), 4929-4942.
APA Ji, Chuanjun., Chen, Yadang., Yang, Zhi Xin., & Wu, Enhua (2022). Spatio-temporal compression for semi-supervised video object segmentation. Visual Computer, 39(10), 4929-4942.
MLA Ji, Chuanjun,et al."Spatio-temporal compression for semi-supervised video object segmentation".Visual Computer 39.10(2022):4929-4942.
Files in This Item:
There are no files associated with this item.
Related Services
Recommend this item
Bookmark
Usage statistics
Export to Endnote
Google Scholar
Similar articles in Google Scholar
[Ji, Chuanjun]'s Articles
[Chen, Yadang]'s Articles
[Yang, Zhi Xin]'s Articles
Baidu academic
Similar articles in Baidu academic
[Ji, Chuanjun]'s Articles
[Chen, Yadang]'s Articles
[Yang, Zhi Xin]'s Articles
Bing Scholar
Similar articles in Bing Scholar
[Ji, Chuanjun]'s Articles
[Chen, Yadang]'s Articles
[Yang, Zhi Xin]'s Articles
Terms of Use
No data!
Social Bookmark/Share
All comments (0)
No comment.
 

Items in the repository are protected by copyright, with all rights reserved, unless otherwise indicated.