Cardiff University | Prifysgol Caerdydd ORCA
Online Research @ Cardiff 
WelshClear Cookie - decide language by browser settings

Meaning maps and saliency models based on deep convolutional neural networks are insensitive to image meaning when predicting human fixations

Pedziwiatr, Marek, Kummerer, Matthias, Wallis, Thomas S.A., Bethge, Matthias and Teufel, Christoph 2019. Meaning maps and saliency models based on deep convolutional neural networks are insensitive to image meaning when predicting human fixations. bioRxiv 10.1101/840256

[img]
Preview
PDF - Submitted Pre-Print Version
Available under License Creative Commons Attribution.

Download (3MB) | Preview

Abstract

Eye movements are vital for human vision, and it is therefore important to understand how observers decide where to look. Meaning maps (MMs), a technique to capture the distribution of semantic importance across an image, have recently been proposed to support the hypothesis that meaning rather than image features guide human gaze. MMs have the potential to be an important tool far beyond eye-movements research. Here, we examine central assumptions underlying MMs. First, we compared the performance of MMs in predicting fixations to saliency models, showing that DeepGaze II – a deep neural network trained to predict fixations based on high-level features rather than meaning – outperforms MMs. Second, we show that whereas human observers respond to changes in meaning induced by manipulating object-context relationships, MMs and DeepGaze II do not. Together, these findings challenge central assumptions underlying the use of MMs to measure the distribution of meaning in images.

Item Type: Article
Date Type: Published Online
Status: Submitted
Schools: Psychology
Publisher: Cold Spring Harbor Laboratory
Date of Acceptance: 14 November 2019
Last Modified: 26 Nov 2019 12:47
URI: http://orca.cf.ac.uk/id/eprint/126877

Actions (repository staff only)

Edit Item Edit Item

Downloads

Downloads per month over past year

View more statistics