Cross-modal Integration and Speech Perception in Schizophrenia:
A Neurophysiological Investigation of the McGurk Effect

McGurk paradigm
The McGurk-MacDonald Illusion

Background:


The McGurk Effect is an illusion that demonstrates the influence of visual information on auditory processing using the context of a speech perception task.


Pairing a video of an actor vocalizing one syllable /fa/ with the audio recording /ba/ normally produces the illusory precept /va/. This illustrates the integration, or fusion, of audio-visual inputs.


Prior studies using the McGurk paradigm conclude that patients with schizophrenia rely less on visual cue information during speech perception.


Exploiting the McGurk illusion may uncover mechanisms of impairment affecting audio-visual integrative abilities.

Research Questions:



How does the temporal overlap of auditory-visual speech components influence integrative ability in schizophrenia?


How do EEG components differ between healthy individuals and patients with schizophrenia?


Can this model be used to accurately predict whether children and young adults exhibting pre-psychotic (prodromal) symptoms will develop schizophrenia?

Behavioral Response

1. Behavioral Response

Fig 1. depicts frequency of /va/ responses in McGurk illusion condition for each group.


HC showed peak sensitivity to illusion with greater a/v overlap, SZ evidenced no effect.


Main effects of diagnosis (p=.02), offset level (p=.005), and a diagnosis x offset interaction (p=.017) were detected.

Auditory ERPs

2a. Auditory ERPs

Fig 2a. depicts auditory ERPs (Cz) in the McGurk illusion condition overlaid by group.


HC evidenced significant attenuation of P200 relative to SZ (p=.017) and enhanced negativity peaking at N400 (p=.023).


Topo plots represent spatial differences (HC- SZ) in activity distribution between groups.

Behavioral Response

2b. Time Frequency Analysis

Fig 2b. depicts total spectral power (Pz) over the time course of McGurk illusion trials.


Increased theta band power was observed at the time of first vocal inflection (1200ms), onset of auditory stimuli (2000ms), and again at time of highest vocal aperture (2300ms) in HC. No related activation was observed in SZ.


The occipital-parietal distribution of activity at 2300ms (300ms post auditory onset), and corresponding divergence.