In the lab, we use laminar fMRI methods to investigate how prior knowledge and other cognitive phenomena influence perceptual processing through neural feedback. The above figure provides an example from one of our recent papers: Using ultra-high resolution fMRI (A), we are able to measure brain responses from separate cortical layers (B). Feed-forward and feedback connections between brain areas target separate layers of a cortical unit (C), meaning we can disentangle feed-forward and feedback computations by examining their organization across layers. For example, perception of illusory shapes is associated with an increased response through top-down feedback in regions of primary visual cortex that receive no bottom-up input  (D). This response is specific to deep cortical layers (E), consistent with feedback connections that avoid the middle input layer, whilst bottom-up activation of the same cortex causes activity throughout all layers. We are currently working on further laminar fMRI projects investigating the laminar computations involved in other cognitive influences on visual processing such as visual working memory, attention, contextual inference and prediction. 

 Relevant papers

 • Lawrence SJD, van Mourik T, Kok P, Koopmans PJ, Norris DG, de Lange FP (2018). Laminar Organization of Working Memory Signals in Human Visual Cortex. Current Biology [pdf][data & code]

 • Kok P, Bains LJ, van Mourik T, Norris DG, de Lange FP (2016). Selective activation of the deep layers of the human primary visual cortex by top-down feedback. Current Biology [pdf]

 • Lawrence SJD, Formisano E, Muckli L, de Lange FP (2017). Laminar fMRI: Applications for cognitive neuroscience. NeuroImage [pdf]

Several studies have shown that perceiving a predicted stimulus is associated with decreased activity compared to an unexpected stimulus. What happens to the neural representation of a predicted stimulus? Does the representation become “sharpened” (and therefore less neurons are involved in processing the stimulus) or is the representation “dampened”, because it was already predicted? Using forward encoding and multivariate pattern analyses, we determine how sensory representations are changed by prior expectation.

Relevant papers:

• Kok P, Jehee JFM, de Lange FP (2012). Less is more: Expectation sharpens representations in the primary visual cortex. Neuron  [pdf]

• Richter D, Ekman M, de Lange FP (2018). Suppressed sensory response to predictable object stimuli throughout the ventral visual stream. Journal of Neuroscience [pdf]

• Kok P, Mostert P, de Lange FP (2017). Prior expectations induce pre-stimulus sensory templates. Proceedings of the National Academy of Sciences [pdf] [data & code]

Numerous studies have shown that during visual working memory, visual cortex is recruited to maintain a representation of the to-be-remembered object. Our lab has even shown that this process is very similar to mental imagery: when participants imagine a stimulus, the activity pattern in visual cortex can be used to classify activity patterns during working memory maintenance. We are now diving into the mechanisms underlying these internal representations by determining in which layer of visual cortex these patterns are most evident. This can provide evidence as to what the contribution of feedback mechanisms to working memory representations are. Similar to working memory and imagery, prediction of an upcoming event can lead to a neural representation of the upcoming stimulus. Using population receptive field mapping, we showed that at specific locations in visual cortex where a stimulus is expected, a response is generated even before that stimulus is presented.

Relevant papers

• Albers AM, Kok P, Toni I, Dijkerman HC, de Lange FP (2013). Shared representations for working memory and mental imagery in early visual cortex. Current Biology [pdf]• Ekman M, Kok P, de Lange FP (2017). Time-compressed preplay of anticipated events in human primary visual cortex. Nature Communications [pdf][data and code]

Our environment is highly structured in both space and time. For instance, our world tends to be stable over short time scales and stimuli that have occurred recently are likely to reoccur. Consequently, our brains can leverage information from the recent past to predict future sensory input. However, sometimes important, unpredicted changes occur, which should not be obliterated by what we expect to experience. In line with these opposite demands of stability and sensitivity, we found that delayed perceptual decisions about recently encountered stimuli are biased towards previous stimuli, facilitating stable representations, whereas the immediate percept of a stimulus is biased away from previous stimuli, increasing sensitivity to small changes in the environment. Furthermore, we found that serial dependencies in perceptual decisions are reflected in activity patterns in primary visual cortex – one of the earliest stages of visual processing. With our ongoing research we aim to understand how predictions are utilized to optimize perceptual decision making in neurotypical and atypical brains.

Figure 1 | Expectations in visual perception. We illustrate some of the ways in which expectations shape perception using an example from Edgar Degas’ celebrated 1878 painting The Rehearsal (left panel). a | Autocorrelation in visual signals. In this excerpt taken from the floor, it is easy to predict the colour of the missing circular patch (2) from the background. Notably, because of the smoothly varying saturation of the background colour, it should be more similar to patch 1 than to patch 3.b | Hierarchical organization of visual information. Viewed through an aperture (left panel), this portion of the image is hard to identify. This is the problem faced by neurons in the early visual system with small receptive fields. Contextual information from a wider aperture (right panel) aids the interpretation of the visual signals. In the visual system, top-down signals from subsequent processing stages, in which receptive fields are larger, may provide contextual information. c | Prior information and priming. Stimuli that are ubiquitous in the visual environment, such as faces, come to be expected and garner dedicated processing mechanisms. Stimuli that have occurred recently are also more likely to reoccur, and repeated stimuli can thus facilitate perception, a phenomenon known as priming. For example, a saccade to the unambiguous face (right panel) will aid subsequent interpretation of the ambiguous face (left panel). Image courtesy of Harvard Art Museums/Fogg Museum, Bequest from the Collection of Maurice Wertheim, Class of 1906, 1951.47, Imaging Department © President and Fellows of Harvard College.

Relevant papers:

• Fritsche M, Mostert P, de Lange FP (2017). Opposite Effects of Recent History on Perception and Decision. Current Biology [pdf] [data & code]

• St John Saaltink E, Kok P, Lau HC, de Lange FP (2016). Serial dependence in perceptual decisions is reflected in activity patterns in primary visual cortex. Journal of Neuroscience [pdf]

• Summerfield C, de Lange FP (2014). Expectation in perceptual decision-making: neural and computational mechanisms. Nature Reviews Neuroscience [pdf]

When processing language, especially on the perceptual level, context is always taken into account; context about the voice characteristics of a speaker, but also about the visual cues that accompany speech. By using the famous McGurk illusion, we showed that auditory perception is altered based on the visual cues associated with the speech sounds, and this effect carries over onto subsequent speech, both neurally and behaviorally. In other words, listeners recalibrate their sound categories based on previously learned context. The other way around, when we process written words, this affects how we perceive visual stimuli. If, for example, participants read a word related to an upward movement, they are faster to detect upward moving stimuli. Interestingly, this effect takes place only when the left hemisphere – the hemisphere that is dominant in language processing – is involved in processing the visual stimuli, suggesting that the neural prediction network operates on a local level.

Relevant papers
• Francken JC, Meijs EL, Hagoort P, van Gaal S, de Lange FP (2015). Exploring the automaticity of language-perception interactions: Effects of attention and awareness. Scientific Reports, 5:17725. pdf
• Francken JC, Kok P, Hagoort P, de Lange FP (2015). The behavioral and neural effects of language on motion perception. Journal of Cognitive Neuroscience, 27, 175-84. pdf
• Francken JC, Meijs EL, Ridderinkhof O, Hagoort P, de Lange FP, van Gaal S (2015). Manipulating word awareness dissociates feed-forward from feedback models of language-perception interactions. Neuroscience of Consciousness, 2015:1. pdf
• Lüttke CS, Ekman M, van Gerven MA, de Lange FP (2016). McGurk illusion recalibrates subsequent auditory perception. Scientific Reports, 6:32891. pdf

We are developing a new research line into the behavioral and neural mechanisms underlying curiosity.

 We are mainly interested in non-instrumental curiosity – curiosity that doesn’t have a direct reward or goal associated with it. We use both explicit (A) and implicit (B) measures to test people’s curiosity. It appears that uncertainty about an outcome is a key determinant of non-instrumental curiosity (C). This suggests that relieving uncertainty – and thus updating our model of the world – is one of the goals of this type of curiosity. Neurally, this is reflected by a greater response in the insula when there is a larger information prediction error (i.e., a larger update of information).

Relevant papers

• van Lieshout L, Vandenbroucke A, Müller N, Cools R, de Lange FP (2018). Induction and relief of curiosity elicit parietal and frontal activity. Journal of Neuroscience, 2816-17. pdf