Towards efficient image and video style transfer via distillation and learnable feature transformation

Huo, Jing, Kong, Meihao, Li, Wenbin, Wu, Jing ORCID: https://orcid.org/0000-0001-5123-9861, Lai, Yu-Kun ORCID: https://orcid.org/0000-0002-2094-5680 and Gao, Yang 2024. Towards efficient image and video style transfer via distillation and learnable feature transformation. Computer Vision and Image Understanding 241 , 103947. 10.1016/j.cviu.2024.103947 Item availability restricted.

PDF - Accepted Post-Print Version Restricted to Repository staff only until 2 February 2025 due to copyright restrictions. Download (20MB)

Abstract

Despite the recent rapid development of neural style transfer, existing style transfer methods are still somewhat inefficient or have a large model size, which limits their application on computational resource limited devices. The major problem lies in that they usually adopt a pre-trained VGG-19 backbone which is relatively large or the feature transformation module is computationally heavy. To address above problems, we propose a DIstillation based Style Transfer framework (called DIST) in conjunction with an efficient feature transformation module for arbitrary image and video style transfer. The distillation module can lead to a highly compressed backbone network, which is 15.95 smaller than the VGG-19 based backbone. The proposed feature transformation is capable of transforming the content features in an extremely efficient feed forward pass. For video style transfer, the above framework is further combined with a temporal consistency regularization loss. Extensive experiments show that the proposed method is superior over the state-of-the-art image and video style transfer methods, even with a much smaller model size.

Item Type:	Article
Date Type:	Publication
Status:	Published
Schools:	Computer Science & Informatics
Publisher:	Elsevier
ISSN:	1077-3142
Date of First Compliant Deposit:	16 March 2024
Date of Acceptance:	25 January 2024
Last Modified:	28 Mar 2024 00:41
URI:	https://orca.cardiff.ac.uk/id/eprint/167296

Actions (repository staff only)

Edit Item