Parsing a mental program: fixation-related brain signatures of unitary operations and routines in natural visual search

Kamienkowski, Juan E.; Varatharajah, Alexander; Sigman, Mariano; Ison, Matias J.

Abstract

Visual search involves a sequence or routine of unitary operations (i.e. fixations) embedded in a larger mental global program. The process can indeed be seen as a program based on a while loop (while the target is not found), a conditional construct (whether the target is matched or not based on specific recognition algorithms) and a decision making step to determine the position of the next searched location based on existent evidence. Recent developments in our ability to co-register brain scalp potentials (EEG) during free eye movements has allowed investigating brain responses related to fixations (fixation-Related Potentials; fERPs), including the identification of sensory and cognitive local EEG components linked to individual fixations. However, the way in which the mental program guiding the search unfolds has not yet been investigated. We performed an EEG and eye tracking co-registration experiment in which participants searched for a target face in natural images of crowds. Here we show how unitary steps of the program are encoded by specific local target detection signatures and how the positioning of each unitary operation within the global search program can be pinpointed by changes in the EEG signal amplitude as well as the signal power in different frequency bands. By simultaneously studying brain signatures of unitary operations and those occurring during the sequence of fixations, our study sheds light into how local and global properties are combined in implementing visual routines in natural tasks.

Citation

Kamienkowski, J. E., Varatharajah, A., Sigman, M., & Ison, M. J. (2018). Parsing a mental program: fixation-related brain signatures of unitary operations and routines in natural visual search. NeuroImage, 183, 73-86. https://doi.org/10.1016/j.neuroimage.2018.08.010

• Journal Article Type: Article
• Acceptance Date: Aug 6, 2018
• Online Publication Date: Aug 8, 2018
• Publication Date: 2018-12
• Deposit Date: Oct 23, 2018
• Publicly Available Date: Aug 9, 2019
• Journal: NeuroImage
• Print ISSN: 1053-8119
• Electronic ISSN: 1095-9572
• Publisher: Elsevier
• Peer Reviewed: Peer Reviewed
• Volume: 183
• Pages: 73-86
• DOI: https://doi.org/10.1016/j.neuroimage.2018.08.010
• Keywords: Visual search; EEG; Eye movements; Attention; Natural scenes; Oscillations
• Public URL: https://nottingham-repository.worktribe.com/output/1051757
• Publisher URL: https://www.sciencedirect.com/science/article/pii/S105381191830702X
• Additional Information: This article is maintained by: Elsevier; Article Title: Parsing a mental program: Fixation-related brain signatures of unitary operations and routines in natural visual search; Journal Title: NeuroImage; CrossRef DOI link to publisher maintained version: https://doi.org/10.1016/j.neuroimage.2018.08.010; Content Type: article; Copyright: © 2018 Elsevier Inc. All rights reserved.
• Contract Date: Oct 23, 2018