Every day we translate the sound waves of speech into meaningful words and sentences. When we do that, we categorize ambiguous sounds into discrete percepts (e.g. phonemes /b/ and /d/). Different speakers can pronounce phonemes differently, but our brain can flexibly adapt to these instabilities, for instance, by taking into account visual information.
Here, we demonstrate that auditory perception can be affected by previously experiencing the McGurk illusion (i.e., the illusory percept of /ADA/ due to hearing an/ABA/ sound and seeing someone say /AGA/ at the same time). Immediately after a McGurk illusion auditory /ABA/ was frequently perceived as /ADA/. The integration of the audiovisual stimulation during a McGurk stimulus was essential– the recalibration effect only occurred when the McGurk illusion was experienced. The effect was not only reflected in a shifted criterion but also in a decreased sensitivity. In other words, /b/ and /d/ became more perceptually similar. Furthermore, the effect was stimulus specific and can be accounted by a Bayesian belief-updating model of perception. We propose that our brain uses visual information (lips movements) to recalibrate auditory perception, narrowing the way that phonemes are perceived in a speaker specific manner.
This paper is now published in Royal Society Open.