UM
Residential Collegefalse
Status即將出版Forthcoming
Depicting Beyond Scores: Advancing Image Quality Assessment Through Multi-modal Language Models
You, Zhiyuan1,2; Li, Zheyuan2,6; Gu, Jinjin3,4; Yin, Zhenfei3,4; Xue, Tianfan1; Dong, Chao2,3,5
2025
Conference Name18th European Conference on Computer Vision, ECCV 2024
Source PublicationLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume15105 LNCS
Pages259-276
Conference Date29 September 2024 to 4 October 2024
Conference PlaceMilan; Italy
PublisherSpringer Science and Business Media Deutschland GmbH
Abstract

We introduce a Depicted image Quality Assessment method (DepictQA), overcoming the constraints of traditional score-based methods. DepictQA allows for detailed, language-based, human-like evaluation of image quality by leveraging Multi-modal Large Language Models (MLLMs). Unlike conventional Image Quality Assessment (IQA) methods relying on scores, DepictQA interprets image content and distortions descriptively and comparatively, aligning closely with humans’ reasoning process. To build the DepictQA model, we establish a hierarchical task framework, and collect a multi-modal IQA training dataset. To tackle the challenges of limited training data and multi-image processing, we propose to use multi-source training data and specialized image tags. These designs result in a better performance of DepictQA  than score-based approaches on multiple benchmarks. Moreover, compared with general MLLMs, DepictQA can generate more accurate reasoning descriptive languages. We also demonstrate that our full-reference dataset can be extended to non-reference applications. These results showcase the research potential of multi-modal IQA methods.

KeywordImage Quality Assessment Multi-modal Language Models
DOI10.1007/978-3-031-72970-6_15
URLView the original
Language英語English
Scopus ID2-s2.0-85210869840
Fulltext Access
Citation statistics
Document TypeConference paper
CollectionUniversity of Macau
Affiliation1.The Chinese University of Hong Kong, Sha Tin, Hong Kong
2.Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Beijing, China
3.Shanghai AI Laboratory, Shanghai, China
4.University of Sydney, Camperdown, Australia
5.Shenzhen University of Advanced Technology, Shenzhen, China
6.University of Macau, Taipa, Macao
Recommended Citation
GB/T 7714
You, Zhiyuan,Li, Zheyuan,Gu, Jinjin,et al. Depicting Beyond Scores: Advancing Image Quality Assessment Through Multi-modal Language Models[C]:Springer Science and Business Media Deutschland GmbH, 2025, 259-276.
APA You, Zhiyuan., Li, Zheyuan., Gu, Jinjin., Yin, Zhenfei., Xue, Tianfan., & Dong, Chao (2025). Depicting Beyond Scores: Advancing Image Quality Assessment Through Multi-modal Language Models. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 15105 LNCS, 259-276.
Files in This Item:
There are no files associated with this item.
Related Services
Recommend this item
Bookmark
Usage statistics
Export to Endnote
Google Scholar
Similar articles in Google Scholar
[You, Zhiyuan]'s Articles
[Li, Zheyuan]'s Articles
[Gu, Jinjin]'s Articles
Baidu academic
Similar articles in Baidu academic
[You, Zhiyuan]'s Articles
[Li, Zheyuan]'s Articles
[Gu, Jinjin]'s Articles
Bing Scholar
Similar articles in Bing Scholar
[You, Zhiyuan]'s Articles
[Li, Zheyuan]'s Articles
[Gu, Jinjin]'s Articles
Terms of Use
No data!
Social Bookmark/Share
All comments (0)
No comment.
 

Items in the repository are protected by copyright, with all rights reserved, unless otherwise indicated.