Spring 2012 Colloquium Series

Numerous studies have shown that, during a bout of face-to-face interaction, multiple patterns of mutual influence unfold between people. For example, conversation partners may come to imitate each other's style of speaking or gesture. Influence can also cut across behaviors, such as when a speaker gestures and smiles while a listener nods in anticipation. These bouts of interaction, stretching just over several seconds or minutes, are constituted by multiple behavioral and linguistic dimensions. A fuller picture of this process will come from an integration of the many channels that are adapted simultaneously during discourse. Here we present a formal characterization and analysis of this mutual influence that permits exploration of a very large number of channels simultaneously: sequential dynamical systems. This approach, derived from widely used methods of network analysis and graph theory, offers a new level of analysis in conceptualizing complex linguistic interaction. We import ideas from network analysis, applied in a wide variety of domains such as social network analysis, brain structure and function, and genomics. The result is a holistic representation of the overall multimodal interaction. This “bird’s-eye view” of discourse offers the view of face-to-face interaction as a loosely coupled, adaptive multimodal network.

Sign-speech bilinguals are fluent in a signed and a spoken language, and as such their two languages do not compete for articulators in the same way that a speech-speech bilingual’s two languages do. The study of sign-speech bilinguals can help us understand the nature of bilingualism, specifically the aspects of the bilingual experience that stem from the representation of two languages or from a modality constraint that forces continual language selection. I review evidence that shows that morphosyntactic elements from a sign-speech bilingual’s two languages are simultaneously active and are difficult to inhibit, that lexical retrieval is difficult for all types of bilinguals, and that the cognitive control advantage observed for speech-speech bilinguals does not extend to sign-speech bilinguals. Taken together, the evidence points to the need to develop models of bilingualism that account for both the representational and the articulatory experience of managing two languages.