Cologne Aachen Gesture Colloquium

Amy L. Hubbard (German Sport University Cologne)

Giving Speech a Hand: Neural Processing of Co-speech Gesture

February 1, 2016

Deutsche Sporthochschule Köln, Am Sportpark Müngersdorf 6, 50933 Köln

A Cooperation between German Sport University Cologne (Hedda Lausberg) and RWTH Aachen University (Irene Mittelberg).

Abstract: Successful social communication involves the integration of simultaneous input from multiple sensory modalities. Here, we used an ecologically valid functional magnetic resonance imaging (fMRI) paradigm to investigate how the brain processes simultaneous communicative cues from visual and auditory modalities, namely, spontaneously produced beat gesture and speech. We examined neural processing of co-speech beat gesture in two groups of neurotypical adults (i.e., native monolingual English speakers and Japanese-English bilinguals) and in two groups of children (i.e., children with Autism Spectrum Disorder and typically developing children). In both adult groups, we observed significantly greater increases in neural activity in regions known to be involved in speech processing when speech was accompanied by beat gesture (Hubbard et al., 2009). In typically developing children, similar to adult subjects, we observed increased activity in a subset of canonical speech processing regions when speech was accompanied by beat gesture. Children with ASD, however, showed no increases in those regions when speech was accompanied by beat gesture; instead, significantly greater activity (over their typically-developing counterparts) was observed in visual cortices (Hubbard et al., 2012). These findings suggest that speech and gesture contribute jointly to communication during social interactions and that neural processes underlying co-speech gesture processing are disrupted in a clinical disorder that is characterized by deficits in social communication.