Faa faa black sheep…
Is lip reading just for the deaf? It turns out that the answer is no. Even people with normal hearing rely on lip reading to decipher what they are hearing. This is known as the McGurk effect (or sometimes as the McGurk-MacDonald effect) since it was first described in the 1976 paper by Harry McGurk and John MacDonald “Hearing lips and seeing voices” (published in Nature, Vol 264(5588), pp. 746–748).
The McGurk effect may be experienced when a video of one phoneme’s production is dubbed with a sound-recording of a different phoneme being spoken. For example, in this video you see a man producing alternatively baa-baa-baa (with a bilabial consonant, where two lips touch) and faa-faa-faa (with a labio-dental consonant, where the lower lip touches the upper teeth). However, the actual sound recorded throughout is only [baa-baa-baa] — and yet, if you look at the man’s face, you hear either [baa] or [faa], depending on what you see!
Similarly, this Youtube video allows you to experience the McGurk effect: close your eyes and you will hear [baa-baa]; open your eyes and you will “hear” [daa-daa]!
This effect suggests that speech perception is multimodal, that is, that it involves information from more than one sensory modality. Sometimes the information from different senses is added up or “averaged”: for example, sometimes the perceived phoneme is neither what your ears hear nor what your eyes see, but a third, intermediate phoneme. For example, a visual [ga] (with a velar consonant, produced in the back of the mouth) combined with an audio [ba] (with a bilabial consonant, produced by the two lips) is often heard as [da] (with an alveolar consonant, who’s place of articulation is midway between labial and velar).
Interestingly, the McGurk effect is very robust: knowing about it seems to have little effect on one’s perception of it. This is different from certain optical illusions, which break down once one ‘sees through’ them.
Is this effect the reason why it is so difficult to follow some contemporary films where the camera shows the person who’s listening and not the person who’s speaking?
Related Posts
Subscribe For Updates
We would love to have you back on Languages Of The World in the future. If you would like to receive updates of our newest posts, feel free to do so using any of your favorite methods below:
This website is a discussion forum dedicated by 