The goal of the Predictive Brain Lab is to understand how the brain forms predictions, and how this facilitates perception and decision-making.
Prediction in Perception
Feedforward and feedback signals
The cortex is a massively recurrent network, characterised by feed-forward and feedback connections between brain areas as well as lateral connections within an area. Feed-forward signals convey ‘bottom-up’ sensory information, whereas feedback signals can modulate this input, based on our goals and expectations. A major goal of our group is to elucidate the interplay between top-down prediction signals and bottom-up input signals within the cortical column, using lamina-resolved neuroimaging techniques.read more
In the lab, we use laminar fMRI methods to investigate how prior knowledge and other cognitive phenomena influence perceptual processing through neural feedback. The above figure provides an example from one of our recent papers: Using ultra-high resolution fMRI (A), we are able to measure brain responses from separate cortical layers (B). Feed-forward and feedback connections between brain areas target separate layers of a cortical unit (C), meaning we can disentangle feed-forward and feedback computations by examining their organization across layers. For example, perception of illusory shapes is associated with an increased response through top-down feedback in regions of primary visual cortex that receive no bottom-up input (D). This response is specific to deep cortical layers (E), consistent with feedback connections that avoid the middle input layer, whilst bottom-up activation of the same cortex causes activity throughout all layers. We are currently working on further laminar fMRI projects investigating the laminar computations involved in other cognitive influences on visual processing such as visual working memory, attention, contextual inference and prediction.
How does prior knowledge change the sensory computations that ultimately result in perception? Does our brain filter out expected signals, dampening their sensory response, or does expectation result in a sharper representation by effectively filtering out the noise? The lab studies this question using both fMRI and MEG.read more
Several studies have shown that perceiving a predicted stimulus is associated with decreased activity compared to an unexpected stimulus. What happens to the neural representation of a predicted stimulus? Does the representation become “sharpened” (and therefore less neurons are involved in processing the stimulus) or is the representation “dampened”, because it was already predicted? Using forward encoding and multivariate pattern analyses, we determine how sensory representations are changed by prior expectation.
Our thoughts and memories are guided by internal representations. Similar to predictions, internal representations are constructed by feedback from higher areas to lower (sensory) areas. To learn more about how these internal representations are generated, we study cognitive processes in which these occur such as working memory and mental imagery.read more
Numerous studies have shown that during visual working memory, visual cortex is recruited to maintain a representation of the to-be-remembered object. Our lab has even shown that this process is very similar to mental imagery: when participants imagine a stimulus, the activity pattern in visual cortex can be used to classify activity patterns during working memory maintenance. We are now diving into the mechanisms underlying these internal representations by determining in which layer of visual cortex these patterns are most evident. This can provide evidence as to what the contribution of feedback mechanisms to working memory representations are. Similar to working memory and imagery, prediction of an upcoming event can lead to a neural representation of the upcoming stimulus. Using population receptive field mapping, we showed that at specific locations in visual cortex where a stimulus is expected, a response is generated even before that stimulus is presented.
• Albers AM, Kok P, Toni I, Dijkerman HC, de Lange FP (2013). Shared representations for working memory and mental imagery in early visual cortex. Current Biology [pdf]• Ekman M, Kok P, de Lange FP (2017). Time-compressed preplay of anticipated events in human primary visual cortex. Nature Communications [pdf][data and code]
Prediction in Cognition
How do we decide what we perceive? Obviously, we base our decisions on the sensory information that enters our brain. However, this sensory information is often uncertain and ambiguous, which complicates perception. The lab studies how the brain forms predictions based on structural regularities in the environment and uses these predictions to aid perceptual decisions under uncertainty.read more
Our environment is highly structured in both space and time. For instance, our world tends to be stable over short time scales and stimuli that have occurred recently are likely to reoccur. Consequently, our brains can leverage information from the recent past to predict future sensory input. However, sometimes important, unpredicted changes occur, which should not be obliterated by what we expect to experience. In line with these opposite demands of stability and sensitivity, we found that delayed perceptual decisions about recently encountered stimuli are biased towards previous stimuli, facilitating stable representations, whereas the immediate percept of a stimulus is biased away from previous stimuli, increasing sensitivity to small changes in the environment. Furthermore, we found that serial dependencies in perceptual decisions are reflected in activity patterns in primary visual cortex – one of the earliest stages of visual processing. With our ongoing research we aim to understand how predictions are utilized to optimize perceptual decision making in neurotypical and atypical brains.
Figure 1 | Expectations in visual perception. We illustrate some of the ways in which expectations shape perception using an example from Edgar Degas’ celebrated 1878 painting The Rehearsal (left panel). a | Autocorrelation in visual signals. In this excerpt taken from the floor, it is easy to predict the colour of the missing circular patch (2) from the background. Notably, because of the smoothly varying saturation of the background colour, it should be more similar to patch 1 than to patch 3.b | Hierarchical organization of visual information. Viewed through an aperture (left panel), this portion of the image is hard to identify. This is the problem faced by neurons in the early visual system with small receptive fields. Contextual information from a wider aperture (right panel) aids the interpretation of the visual signals. In the visual system, top-down signals from subsequent processing stages, in which receptive fields are larger, may provide contextual information. c | Prior information and priming. Stimuli that are ubiquitous in the visual environment, such as faces, come to be expected and garner dedicated processing mechanisms. Stimuli that have occurred recently are also more likely to reoccur, and repeated stimuli can thus facilitate perception, a phenomenon known as priming. For example, a saccade to the unambiguous face (right panel) will aid subsequent interpretation of the ambiguous face (left panel). Image courtesy of Harvard Art Museums/Fogg Museum, Bequest from the Collection of Maurice Wertheim, Class of 1906, 1951.47, Imaging Department © President and Fellows of Harvard College.
In a typical conversation, humans perceive (or produce) about 2-3 words per second. One way to achieve such rapid and robust processing of often-ambiguous linguistic stimuli is to generate predictions about upcoming words. Vice versa, language itself can guide expectations in other domains. In the lab, we study the role of predictions in language processing, and use language as a tool to study prediction in perception.read more
When processing language, especially on the perceptual level, context is always taken into account; context about the voice characteristics of a speaker, but also about the visual cues that accompany speech. By using the famous McGurk illusion, we showed that auditory perception is altered based on the visual cues associated with the speech sounds, and this effect carries over onto subsequent speech, both neurally and behaviorally. In other words, listeners recalibrate their sound categories based on previously learned context. The other way around, when we process written words, this affects how we perceive visual stimuli. If, for example, participants read a word related to an upward movement, they are faster to detect upward moving stimuli. Interestingly, this effect takes place only when the left hemisphere – the hemisphere that is dominant in language processing – is involved in processing the visual stimuli, suggesting that the neural prediction network operates on a local level.
• Francken JC, Meijs EL, Hagoort P, van Gaal S, de Lange FP (2015). Exploring the automaticity of language-perception interactions: Effects of attention and awareness. Scientific Reports, 5:17725. pdf
• Francken JC, Kok P, Hagoort P, de Lange FP (2015). The behavioral and neural effects of language on motion perception. Journal of Cognitive Neuroscience, 27, 175-84. pdf
• Francken JC, Meijs EL, Ridderinkhof O, Hagoort P, de Lange FP, van Gaal S (2015). Manipulating word awareness dissociates feed-forward from feedback models of language-perception interactions. Neuroscience of Consciousness, 2015:1. pdf
• Lüttke CS, Ekman M, van Gerven MA, de Lange FP (2016). McGurk illusion recalibrates subsequent auditory perception. Scientific Reports, 6:32891. pdf
Curiosity is a fundamental biological drive and it is important for many of our daily pursuits and interactions. But why are we curious? And what type of information are we curious about? We believe that curiosity plays a key role in driving us to update our predictions of the world.read more
We are developing a new research line into the behavioral and neural mechanisms underlying curiosity.
We are mainly interested in non-instrumental curiosity – curiosity that doesn’t have a direct reward or goal associated with it. We use both explicit (A) and implicit (B) measures to test people’s curiosity. It appears that uncertainty about an outcome is a key determinant of non-instrumental curiosity (C). This suggests that relieving uncertainty – and thus updating our model of the world – is one of the goals of this type of curiosity. Neurally, this is reflected by a greater response in the insula when there is a larger information prediction error (i.e., a larger update of information).
- van Lieshout L, Vandenbroucke A, Müller N, Cools R, de Lange FP (2018). Induction and relief of curiosity elicit parietal and frontal activity. Journal of Neuroscience, 2816-17. pdf