Article published In:
Interaction StudiesVol. 15:1 (2014) ► pp.38–54
Infant-directed visual prosody
Mothers’ head movements and speech acoustics
Acoustical changes in the prosody of mothers’ speech to infants are distinct and near universal. However, less is known about the visible properties of mothers’ infant-directed (ID) speech, and their relation to speech acoustics. Mothers’ head movements were tracked as they interacted with their infants using ID speech, and compared to movements accompanying their adult-directed (AD) speech. Movement measures along three dimensions of head translation, and three axes of head rotation were calculated. Overall, more head movement was found for ID than AD speech, suggesting that mothers exaggerate their visual prosody in a manner analogous to the acoustical exaggerations in their speech. Regression analyses examined the relation between changing head position and changing acoustical pitch (F0) over time. Head movements and voice pitch were more strongly related in ID speech than in AD speech. When these relations were examined across time windows of different durations, stronger relations were observed for shorter time windows (< 5 sec). However, the particular form of these more local relations did not extend or generalize to longer time windows. This suggests that the multimodal correspondences in speech prosody are variable in form, and occur within limited time spans.
References
Bahrick, L.E., & Pickens, J.N. (
1988). Classification of bimodal English and Spanish language passages by infants. Infant Behavior and Development, 111, 277–296. 
Boersma, P., & Weenick, D. (
2010). Praat: doing phonetics by computer (Version 5.2) [Computer Program].
Brand, R.J., Baldwin, D.A., & Ashburn, L.A. (
2002). Evidence for ‘motionese’: modifications in mothers’ infant-directed action. Developmental Science, 51, 72–83.
Brand, R.J., & Shallcross, W.L. (
2008). Infants prefer motionese to adult-directed action. Developmental Science, 111, 853–861.
Chong, S., Werker, J., Russell, J., & Carroll, J. (
2003). Three facial expressions mothers direct to their infants. Infant and Child Development, 121, 211–232.
Cohn, J., & Tronick, E. (
1988). Mother-infant face-to-face interaction: Influence is bidirectional and unrelated to periodic cycles in either partner’s behavior. Developmental Psychology, 241, 386–392.
Cooper, R.P., & Aslin, R.N. (
1990). Preference for infant-directed speech in the first month after birth. Child Development, 611, 1584–1595.
Davis, C., & Kim, J. (
2006). Audio-visual speech perception off the top of the head. Cognition, 1001, 21–31.
Fernald, A. (
1985). Four-month-old infants prefer to listen to motherese. Infant Behavior and Development, 81, 181–195.
Fernald, A., & Kuhl, P. (
1987). Acoustic determinants of infant preference for motherese speech. Infant Behavior and Development, 101, 279–293.
Fernald, A., & Simon, T. (
1984). Expanded intonation contours in mothers’ speech to newborns. Developmental Psychology, 201, 104–113.
Fernald, A., Taeschner, T., Dunn, J., Papousek, M., de Boysson-Bardies, B., & Fukui, I. (
1989). Across-language study of prosodic modifications in mothers’ and fathers’ speech to preverbal infants. Journal of Child Language, 161, 477–501.
Gogate, L.J., Bahrick, L.E., & Watson, J.D. (
2000). A study of multimodal motherese: the role of temporal synchrony between verbal labels and gestures. Child Development, 711, 878–894.
Gogate, L.J., & Walker-Andrews, A. (
2001). More on developmental dynamics in lexical learning. Developmental Science, 41, 31–37.
Gogate, L.J., Walker-Andrews, A., & Bahrick, L.E. (
2001). The intersensory origins of word comprehension: an ecological-dynamic systems view. Developmental Science, 41, 1–37.
Graf, H.P., Cosatto, E., Strom, V., & Huang, F.J. (
2002). Visual prosody: Facial movements accompanying speech. Paper presented at theFifth IEEE International Conference on Automatic Face and Gesture Recognition (FGR ‘02), Washington, D.C.
Green, J.R., Nip, I.S.B., Wilson, E.M., Mefferd, A.S., & Yunusova, Y. (
2010). Lip movement exaggerations during infant-directed speech. Journal of Speech, Language, and Hearing Research, 531, 1529–1542.
Hadar, U., Steiner, T.J., Grant, E.C., & Rose, F.C. (
1983). Head movement correlates of juncture and stress at sentence level. Language and Speech, 261, 117..
1984). The timing of shifts of head posture during conversation. Human Movement Science, 31, 237–245.
Hadar, U., Steiner, T.J., & Rose, F.C. (
1984). Involvement of head movement in speech production and its implications for language pathology. Advances in Neurology, 42, 247.
Hollich, G., Newman, R.S., & Jusczyk, P. (
2005). Infants’ use of synchronized visual information to separate streams of speech. Child Development, 761, 598–613.
Hollich, G., & Prince, C.G. (
2009). Comparing infants’ preference for correlated audiovisual speech with signal-level computational models. Developmental Science, 121, 379–387.
Kamachi, M., Hill, H., Lander, K., & Vatikiotis-Bateson, E. (
2003). ‘Putting the face to the voice’: Matching identity across modality. Current Biology, 131, 1709–1714.
Kuhl, P.K., & Meltzoff, A. (
1984). The intermodal representation of speech in infants. Infant Behavior and Development, 71, 361–381.
Kuhl, P.K., & Meltzoff, A.N. (
1982). The bimodal perception of speech in infancy. Science, 2181, 1138–1141.
Lewkowicz, D.J., & Hansen-Tift, A.M. (
2012). Infants deploy selective attention to the mouth of a talking face when learning speech. Proceeding of the National Academy of Sciences of the United States of America, 1091, 1431–1436.
Masataka, N. (
1998). Perception of motherese in Japanese sign language by 6-month-old hearing infants. Developmental Psychology, 341, 241–246.
Messinger, D., Mahoor, M.H., Chow, S., & Cohn, J.F. (
2009). Automated measurement of facial express in infant-mother interaction: a pilot study. Infancy, 141, 285–305.
Munhall, K., Jones, J.A., Callan, D.E., Kuratate, T., & Vatikiotis-Bateson, E. (
2004). Visual prosody and speech intelligibility: head movement improves auditory speech perception. Psychological Science, 151, 133–137.
O’Neill, M., Bard, K., Linnell, M., & Fluck, M. (
2005). Maternal gestures with 20-month-old infants in two contexts. Developmental Science, 81, 352–359.
Patterson, M.L., & Werker, J.F. (
2002). Infants’ ability to match dynamic phonetic and gender information in the face and voice. Journal of Experimental Child Psychology, 811, 93–115.
2003). Two‐month‐old infants match phonetic information in lips and voice. Developmental Science, 61, 191–196.
Pegg, J., Werker, J., & McLeod, P. (
1992). Preference for infant-directed over adult-directed speech: Evidence from 7-week-old infants. Infant Behavior and Development, 151, 325–345.
Rosenblum, L.D., Schmuckler, M.A., & Johnson, J.A. (
1997). The McGurk effect in infants. Perception & Psychophysics, 591, 347–357.
Smith, L., & Gasser, M. (
2005). The development of embodied cognition: Six lessons from babies. Artificial Life, 111, 13–29.
Smith, N.A., Gibilisco, C.R., Meisinger, R.E., & Hankey, M. (
2013). Asymmetry in infants’ selective attention to facial features during visual processing of infant-directed speech. Frontiers in Psychology, 41, 601..
Smith, N.A., & Trainor, L.J. (
2008). Infant-directed speech Is modulated by infant feedback. Infancy, 131, 410–420.
Stern, D., Spieker, S., Barnett, R., & MacKain, K. (
1983). The prosody of maternal speech: Infant age and context related changes. Journal of Child Language, 101, 1.
Stern, D., Spieker, S., & MacKain, K. (
1982). Intonation contours as signals in maternal speech to prelinguistic infants. Developmental Psychology, 181, 727–735.
Teinonen, T., Aslin, R., Alku, P., & Csibra, G. (
2008). Visual speech contributes to phonetic learning in 6-month-old infants. Cognition, 1081, 850–855.
Thomas, S.M., & Jordan, T.R. (
2004). Contributions of oral and extraoral facial movement to visual and audiovisual speech perception. Journal of Experimental Psychology: Human Perception and Performance, 301, 873–888.
Thompson, W.F., & Russo, F.A. (
2007). Facing the music. Psychological Science, 181, 756–757.
Walker-Andrews, A. (
1986). Intermodal perception of expressive behaviors: Relation of eye and voice? Developmental Psychology, 221, 373–377.
Weikum, W.M., Vouloumanos, A., Navarra, J., Soto-Faraco, S., Sebastián-Gallés, N., & Werker, J.F. (
2007). Visual language discrimination in infancy. Science, 3161, 1159.
Werker, J.F., Pegg, J., & McLeod, P. (
1994). A cross-language investigation of infant preference for infant-directed communication. Infant Behavior and Development, 171, 323–333.
Yehia, H., Kuratate, T., & Vatikiotis-Bateson, E. (
2002). Linking facial animation, head motion and speech acoustics. Journal of Phonetics, 301, 555–568.
Yehia, H., Rubin, P., & Vatikiotis-Bateson, E. (
1998). Quantitative association of vocal-tract and facial behavior. Speech Communication, 261, 23–43.
Cited by
Cited by 9 other publications
Bessolles, Manon, Christelle Dodane, Bahia Guellai & Hélène Cochet
2024. L'enfant dans son environnement. In L'enfant dans son environnement, ► pp. 149 ff. 
Cartmill, Erica A.
2022. Gesture. Annual Review of Anthropology 51:1 ► pp. 455 ff.
Englund, Nunne & Dawn M. Behne
2020. Perception of audiovisual infant directed speech. Scandinavian Journal of Psychology 61:2 ► pp. 218 ff.
Gogate, Lakshmi
2016. Development of Early Multisensory Perception and Communication: From Environmental and Behavioral to Neural Signatures. Developmental Neuropsychology 41:5-8 ► pp. 269 ff.
Gogate, Lakshmi, Madhavilatha Maganti & Lorraine E. Bahrick
2015. Cross-cultural evidence for multimodal motherese: Asian Indian mothers’ adaptive use of synchronous words and gestures. Journal of Experimental Child Psychology 129 ► pp. 110 ff.
Hübscher, Iris & Pilar Prieto
2019. Gestural and Prosodic Development Act as Sister Systems and Jointly Pave the Way for Children’s Sociopragmatic Development. Frontiers in Psychology 10
Oakes, Lisa M., Michaela C. DeBolt, Aaron G. Beckner, Annika T. Voss & Lisa M. Cantrell
2021. Infant Eye Gaze While Viewing Dynamic Faces. Brain Sciences 11:2 ► pp. 231 ff.
Smith, Nicholas A., Nicole A. Folland, Diana M. Martinez & Laurel J. Trainor
2017. Multisensory object perception in infancy: 4-month-olds perceive a mistuned harmonic as a separate auditory and visual object. Cognition 164 ► pp. 1 ff.
Trehub, Sandra E., Judy Plantinga & Frank A. Russo
2016. Maternal Vocal Interactions with Infants: Reciprocal Visual Influences. Social Development 25:3 ► pp. 665 ff.
This list is based on CrossRef data as of 31 march 2024. Please note that it may not be complete. Sources presented here have been supplied by the respective publishers. Any errors therein should be reported to them.