Residential College | false |
Status | 已發表Published |
Progressive normalizing flow with learnable spectrum transform for style transfer | |
He, Zixuan1; Huang, Guoheng1; Yuan, Xiaochen2; Zhong, Guo4; Pun, Chi Man3; Zeng, Yiwen1 | |
2024-01-25 | |
Source Publication | Knowledge-Based Systems |
ISSN | 0950-7051 |
Volume | 284Pages:111277 |
Abstract | Most current style transfer models are designed as encoder–decoder structures. Some encoding operations, such as downsampling and pooling, cause a loss of image details. If the encoder and decoder are not compatible, it can also introduce distortion. Reversible neural networks have demonstrated their superior power in lossless projection. However, since the inputs and outputs of neural flows are holistic features, merely the high-level features can be utilized for image generation through reverse inference. These high-level features emphasize the image style more, leading to the generated results easily losing content details and producing abstract colors. To address the above issues, we propose LSTFlow, the first progressive reversible neural network capable of feature decomposition. First, LSTFlow incorporates our proposed reversible Learnable Spectrum Transform (LST), which can dynamically decompose the feature into feature spectrum and recover them losslessly. LSTFlow can retain more details by enabling multi-level features to be fused in backward inference. Second, we propose a Progressive Flow Stylization Strategy (PFSS) to balance the model's emphasis between content and style and enhance the color perception. Forward inference based PFSS is carried out progressively, while the backward inference focuses on progressive generation. To demonstrate the effectiveness of our proposed method, we conducted comparative experiments with seven other state-of-the-art algorithms. The stylized effects are evaluated in terms of visual effects and quantitative indicators. The experiments show that the lightest LSTFlow performs the best in SSIM, Color Entropy, Color Uniformity and FID indicators and outperforms state-of-the-art methods. |
Keyword | Feature Decomposition Feature Spectrum Neural Flow Progressive Stylization Reversible Neural Network Style Transfer |
DOI | 10.1016/j.knosys.2023.111277 |
URL | View the original |
Indexed By | SCIE |
Language | 英語English |
WOS Research Area | Computer Science |
WOS Subject | Computer Science, Artificial Intelligence |
WOS ID | WOS:001142046200001 |
Scopus ID | 2-s2.0-85180752619 |
Fulltext Access | |
Citation statistics | |
Document Type | Journal article |
Collection | DEPARTMENT OF COMPUTER AND INFORMATION SCIENCE |
Corresponding Author | Huang, Guoheng |
Affiliation | 1.School of Computer Science and Technology, Guangdong University of Technology, Guangzhou, 510006, China 2.Faculty of Applied Sciences, Macao Polytechnic University, 999078, China 3.Department of Computer and Information Science, University of Macau, 999078, China 4.School of Information Science and Technology, Guangdong University of Foreign Studies, Guangzhou, 510006, China |
Recommended Citation GB/T 7714 | He, Zixuan,Huang, Guoheng,Yuan, Xiaochen,et al. Progressive normalizing flow with learnable spectrum transform for style transfer[J]. Knowledge-Based Systems, 2024, 284, 111277. |
APA | He, Zixuan., Huang, Guoheng., Yuan, Xiaochen., Zhong, Guo., Pun, Chi Man., & Zeng, Yiwen (2024). Progressive normalizing flow with learnable spectrum transform for style transfer. Knowledge-Based Systems, 284, 111277. |
MLA | He, Zixuan,et al."Progressive normalizing flow with learnable spectrum transform for style transfer".Knowledge-Based Systems 284(2024):111277. |
Files in This Item: | There are no files associated with this item. |
Items in the repository are protected by copyright, with all rights reserved, unless otherwise indicated.
Edit Comment