Cover not available
Article published In:
Interaction Studies
Vol. 15:1 (2014) ► pp.38–54

Infant-directed visual prosody

Mothers’ head movements and speech acoustics

Acoustical changes in the prosody of mothers’ speech to infants are distinct and near universal. However, less is known about the visible properties of mothers’ infant-directed (ID) speech, and their relation to speech acoustics. Mothers’ head movements were tracked as they interacted with their infants using ID speech, and compared to movements accompanying their adult-directed (AD) speech. Movement measures along three dimensions of head translation, and three axes of head rotation were calculated. Overall, more head movement was found for ID than AD speech, suggesting that mothers exaggerate their visual prosody in a manner analogous to the acoustical exaggerations in their speech. Regression analyses examined the relation between changing head position and changing acoustical pitch (F0) over time. Head movements and voice pitch were more strongly related in ID speech than in AD speech. When these relations were examined across time windows of different durations, stronger relations were observed for shorter time windows (< 5 sec). However, the particular form of these more local relations did not extend or generalize to longer time windows. This suggests that the multimodal correspondences in speech prosody are variable in form, and occur within limited time spans.
Published online: 10 June 2014
DOI logo
https://doi.org/10.1075/is.15.1.02smi
References (47)
Bahrick, L.E., & Pickens, J.N. (1988). Classification of bimodal English and Spanish language passages by infants. Infant Behavior and Development, 111, 277–296. DOI logoGoogle Scholar
Boersma, P., & Weenick, D. (2010). Praat: doing phonetics by computer (Version 5.2) [Computer Program].
Brand, R.J., Baldwin, D.A., & Ashburn, L.A. (2002). Evidence for ‘motionese’: modifications in mothers’ infant-directed action. Developmental Science, 51, 72–83. DOI logoGoogle Scholar
Brand, R.J., & Shallcross, W.L. (2008). Infants prefer motionese to adult-directed action. Developmental Science, 111, 853–861. DOI logoGoogle Scholar
Chong, S., Werker, J., Russell, J., & Carroll, J. (2003). Three facial expressions mothers direct to their infants. Infant and Child Development, 121, 211–232. DOI logoGoogle Scholar
Cohn, J., & Tronick, E. (1988). Mother-infant face-to-face interaction: Influence is bidirectional and unrelated to periodic cycles in either partner’s behavior. Developmental Psychology, 241, 386–392. DOI logoGoogle Scholar
Cooper, R.P., & Aslin, R.N. (1990). Preference for infant-directed speech in the first month after birth. Child Development, 611, 1584–1595. DOI logoGoogle Scholar
Davis, C., & Kim, J. (2006). Audio-visual speech perception off the top of the head. Cognition, 1001, 21–31. DOI logoGoogle Scholar
Fernald, A. (1985). Four-month-old infants prefer to listen to motherese. Infant Behavior and Development, 81, 181–195. DOI logoGoogle Scholar
Fernald, A., & Kuhl, P. (1987). Acoustic determinants of infant preference for motherese speech. Infant Behavior and Development, 101, 279–293. DOI logoGoogle Scholar
Fernald, A., & Simon, T. (1984). Expanded intonation contours in mothers’ speech to newborns. Developmental Psychology, 201, 104–113. DOI logoGoogle Scholar
Fernald, A., Taeschner, T., Dunn, J., Papousek, M., de Boysson-Bardies, B., & Fukui, I. (1989). Across-language study of prosodic modifications in mothers’ and fathers’ speech to preverbal infants. Journal of Child Language, 161, 477–501. DOI logoGoogle Scholar
Gogate, L.J., Bahrick, L.E., & Watson, J.D. (2000). A study of multimodal motherese: the role of temporal synchrony between verbal labels and gestures. Child Development, 711, 878–894. DOI logoGoogle Scholar
Gogate, L.J., & Walker-Andrews, A. (2001). More on developmental dynamics in lexical learning. Developmental Science, 41, 31–37. DOI logoGoogle Scholar
Gogate, L.J., Walker-Andrews, A., & Bahrick, L.E. (2001). The intersensory origins of word comprehension: an ecological-dynamic systems view. Developmental Science, 41, 1–37. DOI logoGoogle Scholar
Graf, H.P., Cosatto, E., Strom, V., & Huang, F.J. (2002). Visual prosody: Facial movements accompanying speech. Paper presented at theFifth IEEE International Conference on Automatic Face and Gesture Recognition (FGR ‘02), Washington, D.C.
Green, J.R., Nip, I.S.B., Wilson, E.M., Mefferd, A.S., & Yunusova, Y. (2010). Lip movement exaggerations during infant-directed speech. Journal of Speech, Language, and Hearing Research, 531, 1529–1542. DOI logoGoogle Scholar
Hadar, U., Steiner, T.J., Grant, E.C., & Rose, F.C. (1983). Head movement correlates of juncture and stress at sentence level. Language and Speech, 261, 117.. DOI logoGoogle Scholar
1984). The timing of shifts of head posture during conversation. Human Movement Science, 31, 237–245. DOI logoGoogle Scholar
Hadar, U., Steiner, T.J., & Rose, F.C. (1984). Involvement of head movement in speech production and its implications for language pathology. Advances in Neurology, 42, 247.Google Scholar
Hollich, G., Newman, R.S., & Jusczyk, P. (2005). Infants’ use of synchronized visual information to separate streams of speech. Child Development, 761, 598–613. DOI logoGoogle Scholar
Hollich, G., & Prince, C.G. (2009). Comparing infants’ preference for correlated audiovisual speech with signal-level computational models. Developmental Science, 121, 379–387. DOI logoGoogle Scholar
Kamachi, M., Hill, H., Lander, K., & Vatikiotis-Bateson, E. (2003). ‘Putting the face to the voice’: Matching identity across modality. Current Biology, 131, 1709–1714. DOI logoGoogle Scholar
Kuhl, P.K., & Meltzoff, A. (1984). The intermodal representation of speech in infants. Infant Behavior and Development, 71, 361–381. DOI logoGoogle Scholar
Kuhl, P.K., & Meltzoff, A.N. (1982). The bimodal perception of speech in infancy. Science, 2181, 1138–1141. DOI logoGoogle Scholar
Lewkowicz, D.J., & Hansen-Tift, A.M. (2012). Infants deploy selective attention to the mouth of a talking face when learning speech. Proceeding of the National Academy of Sciences of the United States of America, 1091, 1431–1436. DOI logoGoogle Scholar
Masataka, N. (1998). Perception of motherese in Japanese sign language by 6-month-old hearing infants. Developmental Psychology, 341, 241–246. DOI logoGoogle Scholar
Messinger, D., Mahoor, M.H., Chow, S., & Cohn, J.F. (2009). Automated measurement of facial express in infant-mother interaction: a pilot study. Infancy, 141, 285–305. DOI logoGoogle Scholar
Munhall, K., Jones, J.A., Callan, D.E., Kuratate, T., & Vatikiotis-Bateson, E. (2004). Visual prosody and speech intelligibility: head movement improves auditory speech perception. Psychological Science, 151, 133–137. DOI logoGoogle Scholar
O’Neill, M., Bard, K., Linnell, M., & Fluck, M. (2005). Maternal gestures with 20-month-old infants in two contexts. Developmental Science, 81, 352–359. DOI logoGoogle Scholar
Patterson, M.L., & Werker, J.F. (2002). Infants’ ability to match dynamic phonetic and gender information in the face and voice. Journal of Experimental Child Psychology, 811, 93–115. DOI logoGoogle Scholar
2003). Two‐month‐old infants match phonetic information in lips and voice. Developmental Science, 61, 191–196. DOI logoGoogle Scholar
Pegg, J., Werker, J., & McLeod, P. (1992). Preference for infant-directed over adult-directed speech: Evidence from 7-week-old infants. Infant Behavior and Development, 151, 325–345. DOI logoGoogle Scholar
Rosenblum, L.D., Schmuckler, M.A., & Johnson, J.A. (1997). The McGurk effect in infants. Perception & Psychophysics, 591, 347–357. DOI logoGoogle Scholar
Smith, L., & Gasser, M. (2005). The development of embodied cognition: Six lessons from babies. Artificial Life, 111, 13–29. DOI logoGoogle Scholar
Smith, N.A., Gibilisco, C.R., Meisinger, R.E., & Hankey, M. (2013). Asymmetry in infants’ selective attention to facial features during visual processing of infant-directed speech. Frontiers in Psychology, 41, 601.. DOI logoGoogle Scholar
Smith, N.A., & Trainor, L.J. (2008). Infant-directed speech Is modulated by infant feedback. Infancy, 131, 410–420. DOI logoGoogle Scholar
Stern, D., Spieker, S., Barnett, R., & MacKain, K. (1983). The prosody of maternal speech: Infant age and context related changes. Journal of Child Language, 101, 1. DOI logoGoogle Scholar
Stern, D., Spieker, S., & MacKain, K. (1982). Intonation contours as signals in maternal speech to prelinguistic infants. Developmental Psychology, 181, 727–735. DOI logoGoogle Scholar
Teinonen, T., Aslin, R., Alku, P., & Csibra, G. (2008). Visual speech contributes to phonetic learning in 6-month-old infants. Cognition, 1081, 850–855. DOI logoGoogle Scholar
Thomas, S.M., & Jordan, T.R. (2004). Contributions of oral and extraoral facial movement to visual and audiovisual speech perception. Journal of Experimental Psychology: Human Perception and Performance, 301, 873–888. DOI logoGoogle Scholar
Thompson, W.F., & Russo, F.A. (2007). Facing the music. Psychological Science, 181, 756–757. DOI logoGoogle Scholar
Walker-Andrews, A. (1986). Intermodal perception of expressive behaviors: Relation of eye and voice? Developmental Psychology, 221, 373–377. DOI logoGoogle Scholar
Weikum, W.M., Vouloumanos, A., Navarra, J., Soto-Faraco, S., Sebastián-Gallés, N., & Werker, J.F. (2007). Visual language discrimination in infancy. Science, 3161, 1159. DOI logoGoogle Scholar
Werker, J.F., Pegg, J., & McLeod, P. (1994). A cross-language investigation of infant preference for infant-directed communication. Infant Behavior and Development, 171, 323–333. DOI logoGoogle Scholar
Yehia, H., Kuratate, T., & Vatikiotis-Bateson, E. (2002). Linking facial animation, head motion and speech acoustics. Journal of Phonetics, 301, 555–568. DOI logoGoogle Scholar
Yehia, H., Rubin, P., & Vatikiotis-Bateson, E. (1998). Quantitative association of vocal-tract and facial behavior. Speech Communication, 261, 23–43. DOI logoGoogle Scholar
Cited by (9)

Cited by nine other publications

Bessolles, Manon, Christelle Dodane, Bahia Guellai & Hélène Cochet
2024. L'enfant dans son environnement. In L'enfant dans son environnement, ► pp. 149 ff. DOI logo
Cartmill, Erica A.
2022. Gesture. Annual Review of Anthropology 51:1 ► pp. 455 ff. DOI logo
Oakes, Lisa M., Michaela C. DeBolt, Aaron G. Beckner, Annika T. Voss & Lisa M. Cantrell
2021. Infant Eye Gaze While Viewing Dynamic Faces. Brain Sciences 11:2 ► pp. 231 ff. DOI logo
Englund, Nunne & Dawn M. Behne
2020. Perception of audiovisual infant directed speech. Scandinavian Journal of Psychology 61:2 ► pp. 218 ff. DOI logo
Hübscher, Iris & Pilar Prieto
2019. Gestural and Prosodic Development Act as Sister Systems and Jointly Pave the Way for Children’s Sociopragmatic Development. Frontiers in Psychology 10 DOI logo
Smith, Nicholas A., Nicole A. Folland, Diana M. Martinez & Laurel J. Trainor
2017. Multisensory object perception in infancy: 4-month-olds perceive a mistuned harmonic as a separate auditory and visual object. Cognition 164 ► pp. 1 ff. DOI logo
Gogate, Lakshmi
2016. Development of Early Multisensory Perception and Communication: From Environmental and Behavioral to Neural Signatures. Developmental Neuropsychology 41:5-8 ► pp. 269 ff. DOI logo
Trehub, Sandra E., Judy Plantinga & Frank A. Russo
2016. Maternal Vocal Interactions with Infants: Reciprocal Visual Influences. Social Development 25:3 ► pp. 665 ff. DOI logo
Gogate, Lakshmi, Madhavilatha Maganti & Lorraine E. Bahrick
2015. Cross-cultural evidence for multimodal motherese: Asian Indian mothers’ adaptive use of synchronous words and gestures. Journal of Experimental Child Psychology 129 ► pp. 110 ff. DOI logo

This list is based on CrossRef data as of 12 september 2024. Please note that it may not be complete. Sources presented here have been supplied by the respective publishers. Any errors therein should be reported to them.