Visual quality assessment for super-resolved images: database and method

Zhou, Fei; Yao, Rongguo; Liu, Bozhi; Qiu, Guoping

Abstract

Image super-resolution (SR) has been an active re-search problem which has recently received renewed interest due to the introduction of new technologies such as deep learning. However, the lack of suitable criteria to evaluate the SR perfor-mance has hindered technology development. In this paper, we fill a gap in the literature by providing the first publicly available database as well as a new image quality assessment (IQA) method specifically designed for assessing the visual quality of su-per-resolved images (SRIs). In constructing the Quality Assess-ment Database for SRIs (QADS), we carefully selected 20 refer-ence images and created 980 SRIs using 21 image SR methods. Mean opinion score (MOS) for these SRIs are collected through 100 individuals participating a suitably designed psychovisual experiment. Extensive numerical and statistical analysis is per-formed to show that the MOS of QADS has excellent suitability and reliability. The psychovisual experiment has led to the dis-covery that, unlike distortions encountered in other IQA data-bases, artifacts of the SRIs degenerate the image structure as well as image texture. Moreover, the structural and textural degener-ations have distinctive perceptual properties. Based on these in-sights, we propose a novel method to assess the visual quality of SRIs by separately considering the structural and textural com-ponents of images. Observing that textural degenerations are mainly attributed to dissimilar texture or checkerboard artifacts, we propose to measure the changes of textural distributions. We also observe that structural degenerations appear as blurring and jaggies artifacts in SRIs and develop separate similarity measures for different types of structural degenerations. A new pooling mechanism is then used to fuse the different similarities together to give the final quality score for an SRI. Experiments conducted on the QADS demonstrate that our method significantly outper-forms classical as well as current state-of-the-art IQA methods.

Citation

Zhou, F., Yao, R., Liu, B., & Qiu, G. (2019). Visual quality assessment for super-resolved images: database and method. IEEE Transactions on Image Processing, 28(7), 3528-3541. https://doi.org/10.1109/tip.2019.2898638

• Journal Article Type: Article
• Acceptance Date: Feb 12, 2019
• Online Publication Date: Feb 12, 2019
• Publication Date: Feb 12, 2019
• Deposit Date: Feb 27, 2019
• Publicly Available Date: Feb 27, 2019
• Journal: IEEE Transactions on Image Processing
• Print ISSN: 1057-7149
• Electronic ISSN: 1941-0042
• Publisher: Institute of Electrical and Electronics Engineers
• Peer Reviewed: Peer Reviewed
• Volume: 28
• Issue: 7
• Pages: 3528-3541
• DOI: https://doi.org/10.1109/tip.2019.2898638
• Keywords: Full reference; Image database; Image quality assessment; Image super resolution
• Public URL: https://nottingham-repository.worktribe.com/output/1589699
• Publisher URL: https://ieeexplore.ieee.org/document/8640853
• Additional Information: © 2019 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.
• Contract Date: Feb 27, 2019