Advancing Saliency Ranking with Human Fixations: Dataset, Models and Benchmarks

Deng, Bowen; Song, Siyang; French, Andrew P.; Schluppeck, Denis; Pound, Michael P.

Abstract

Saliency ranking detection (SRD) has emerged as a challenging task in computer vision, aiming not only to identify salient objects within images but also to rank them based on their degree of saliency. Existing SRD datasets have been created primarily using mouse-trajectory data, which inadequately captures the intricacies of human visual perception. Addressing this gap, this paper introduces the first large-scale SRD dataset, SIFR, constructed using genuine human fixation data, thereby aligning more closely with real visual perceptual processes. To establish a baseline for this dataset, we propose QAGNet, a novel model that leverages salient instance query features from a transformer detector within a tri-tiered nested graph. Through extensive experiments, we demonstrate that our approach outperforms existing state-of-the-art methods across two widely used SRD datasets and our newly proposed dataset. Code and dataset are available at https://github.com/EricDengbowen/QAGNet.

Citation

Deng, B., Song, S., French, A. P., Schluppeck, D., & Pound, M. P. (2024, June). Advancing Saliency Ranking with Human Fixations: Dataset, Models and Benchmarks. Presented at Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA

• Presentation Conference Type: Edited Proceedings
• Conference Name: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition
• Start Date: Jun 16, 2024
• End Date: Jun 22, 2024
• Acceptance Date: Feb 27, 2024
• Online Publication Date: Sep 16, 2024
• Publication Date: Sep 16, 2024
• Deposit Date: Mar 6, 2025
• Publicly Available Date: Mar 11, 2025
• Publisher: Institute of Electrical and Electronics Engineers
• Peer Reviewed: Peer Reviewed
• Pages: 28348-28357
• Series ISSN: 2575-7075
• Book Title: 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
• ISBN: 979-8-3503-5301-3
• DOI: https://doi.org/10.1109/cvpr52733.2024.02678
• Public URL: https://nottingham-repository.worktribe.com/output/40250376
• Publisher URL: https://ieeexplore.ieee.org/document/10656059
• Other Repo URL: https://openaccess.thecvf.com/content/CVPR2024/papers/Deng_Advancing_Saliency_Ranking_with_Human_Fixations_Dataset_Models_and_Benchmarks_CVPR_2024_paper.pdf