Chapter published in:
Linguistic Foundations of Narration in Spoken and Sign Languages
Edited by Annika Hübl and Markus Steinbach
[Linguistik Aktuell/Linguistics Today 247] 2018
► pp. 15–40

A new technique for analyzing narrative prosodic effects in sign languages using motion capture technology

| Purdue University
| Purdue University
The present paper addresses prosody at a sentence level analysis within short narratives, developing a novel method based on a combination of data. Our first objective, then, is to establish the validity of our new method by confirming the findings of previous reports on Phrase Final Lengthening. Our second objective is to further extend the validity of the new method by confirming a prior claim regarding the role of increased velocity for signs that are stressed in sentences. Finally, we report new results in the form of patterns across signs within sentences using our new method.
Article outline
  • 1.Introduction
    • 1.1Perceptual studies of prosody and fluency
    • 1.2Production studies of fluency and prosody
    • 1.3Prior kinematic work
      • 1.3.1Prior motion capture work showing sign lowering
      • 1.3.2Prior motion capture work showing verb class differences
      • 1.3.3Prior motion capture work showing phrase final lengthening
      • 1.3.4Prior motion capture work showing stress marking
  • 2.The present study
    • 2.1Procedure
      • 2.1.1Motion capture method for current report
      • 2.1.2Coding for non-kinematic linguistic and perceptual variables
      • 2.1.3Coding for kinematic variables
    • 2.2Analyses
    • 2.3Results
      • 2.3.1Kinematic analysis by sentence position
      • 2.3.2Kinematic analysis of stress
      • 2.3.3More general test of relationships among measured variables and weight
    • 2.4Summary
  • 3.Conclusion
  • Acknowledgements
  • Notes
  • References
Published online: 25 May 2018
https://doi.org/10.1075/la.247.02wil
References
Allen, G. D., Wilbur, R. B. & Schick, B.
1991 Aspects of rhythm in American Sign Language. Sign Language Studies 72: 297–320.CrossrefGoogle Scholar
Boyes Braem, P.
1999 Rhythmic temporal patterns in the signing of deaf early and later learners of Swiss German Sign Language. Language & Speech 42: 177–208.CrossrefGoogle Scholar
Brentari, D.
1998 A Prosodic Model of Sign Language Phonology. Cambridge MA: The MIT Press.Google Scholar
Brentari, D. & Crossley, L.
2002 Prosody on the hands and face: Evidence from American Sign Language. Sign Language & Linguistics 5: 105–130.CrossrefGoogle Scholar
Brentari, D., Nadolske, M. & Wolford, G.
2012 Can experience with co-speech gesture influence the prosody of a sign language? Sign language prosodic cues in bimodal bilinguals. Bilingualism: Language and Cognition 15(2): 402–412.CrossrefGoogle Scholar
Brentari, D., González, C., Seidl, A. & Wilbur, R. B.
2011 Sensitivity to visual prosodic cues in signers and nonsigners. Language & Speech 54: 49–72.CrossrefGoogle Scholar
González, C.
2011 Perception of Prosody in American Sign Language. PhD dissertation, Purdue University.Google Scholar
Green, K.
1984 Sign boundaries in American Sign Language. Sign Language Studies 42: 65–91.CrossrefGoogle Scholar
Grosjean, F. & Lane, H.
1977 Pauses and syntax in American Sign Language. Cognition 5: 101–117.CrossrefGoogle Scholar
Grosvald, M. A.
2009 Long-distance Coarticulation: A Production and Perception Study of English and American Sign Language. PhD dissertation, University of California at Davis.Google Scholar
Kantor, R.
1978 Identifying native and second-language signers. In Sign Language Research, R. B. Wilbur (ed), Special issue of Communication and Cognition 11(1): 39–55.Google Scholar
Kröger, B. J., Birkholz, P., Kannampuzha, J., Kaufmann, E. & Mittelberg, I.
2011 Movements and holds in fluent sentence production of American Sign Language: The action-based approach. Cognitive Computation 3(3): 449–465.CrossrefGoogle Scholar
Liddell, S. K.
1977 An Investigation into the Syntactic Structure of American Sign Language. PhD dissertation, University of California, San Diego.Google Scholar
Lupton, L. K.
1998 Fluency in American Sign Language. Journal of Deaf Studies and Deaf Education 3(4): 320–328.CrossrefGoogle Scholar
Lupton, L. K. & Zelaznik, H.
1990 Motor learning in sign language students. Sign Language Studies 67: 153–174.CrossrefGoogle Scholar
Weast, T.
2008 Questions in American Sign Language: A Quantitative Analysis of Raised and Lowered Eyebrows. Phd dissertation, University of Texas, Arlington, U.S.A.Google Scholar
Malaia, E.
2014 It still isn’t over: Event boundaries in language and perception. Language and Linguistics Compass 8(3): 89–98.CrossrefGoogle Scholar
2017 Current and future methodologies for quantitative analysis of information transfer in sign language and gesture data. Behavioral and Brain Sciences 40.Google Scholar
Malaia, E., Borneman, J. D. & Wilbur, R. B.
2016 Assessment of information content in visual signal: analysis of optical flow fractal complexity. Visual Cognition 24(3): 246–251.CrossrefGoogle Scholar
2017 Information transfer capacity of articulators in American Sign Language. Language and Speech. doi:Crossref.Google Scholar
Malaia, E., Ranaweera, R., Wilbur R. B. & Talavage, T. M.
2012 Neural representation of event structure in American Sign Language: fMRI comparison of cortical activations in deaf signers and hearing non-signers. Neuroimage 59: 4094–4101.CrossrefGoogle Scholar
Malaia, E. & Wilbur, R. B.
2012a Kinematic signatures of telic and atelic events in ASL predicates. Language & Speech 55(3): 407–421.CrossrefGoogle Scholar
2012b Telicity expression in the visual modality. In Telicity, Change, and State: A Cross-categorial View of Event Structure, V. Demonte & L. McNally (eds), 122–136. Oxford: OUP.CrossrefGoogle Scholar
Malaia, E., Wilbur, R. B. & Milković, M.
2013 Kinematic parameters of signed verbs at the morpho-phonology interface. Journal of Speech, Language and Hearing Research 56: 1677–1688.CrossrefGoogle Scholar
Malaia, E., Wilbur, R. B. & Weber-Fox, C.
2013 Event end-point primes the undergoer argument: Neurobiological bases of event structure processing. In Studies in the Composition and Decomposition of Event Predicates, B. Arsenijevic, B. Gehrke & R. Marin (eds), 231–248. Dordrecht: Springer.CrossrefGoogle Scholar
Mauk, C. E.
2003 Undershoot in Two Modalities: Evidence from Fast Speech and Fast Signing. PhD dissertation, University of Texas at Austin.Google Scholar
Mauk, C. E. & Tyrone, M. E.
2008 Sign lowering as phonetic reduction in American Sign Language. In Proceedings of the 2008 International Seminar on Speech Production, R. Sock, S. Fuchs & Y. Laprie (eds), 185–188. http://​issp2008​.loria​.fr​/proceedings​.htmlGoogle Scholar
2012 Location in ASL: Insights from phonetic variation. Sign Language & Linguistics 15(1): 128–146.CrossrefGoogle Scholar
McDonald, J., Wolfe, R., Wilbur, R. B., Moncrief, R., Malaia, E., Fujimoto, S., Baowidan, S. & Stec, J.
2016 A new tool to facilitate prosodic analysis of motion capture data and a data-driven technique for the improvement of avatar motion. In Proceedings of the Seventh Workshop on the Representation and Processing of Sign Languages: Corpus Mining (LREC-2016), 153–158, Portorož, Slovenia.Google Scholar
Milković, M.
2011 Verb Classes in Croatian Sign Language (HZJ): Syntactic and Semantic Properties PhD dissertation, University of Zagreb, Croatia.Google Scholar
Nespor, M. & Sandler, W.
1999 Prosody in Israeli Sign Language. Language and Speech 42: 143–176.CrossrefGoogle Scholar
Ormel, E. & Crasborn, O.
2012 Prosodic correlates of sentences in signed languages: A literature review and suggestions for new types of studies. Sign Language Studies 12(2): 279–315.CrossrefGoogle Scholar
Pfeiffer, T.
2013 Documentation of gestures with motion capture. In Body – Language – Communication, C. Müller, A. Cienki, E. Fricke, H. S. Ladewig, D. McNeill & S. Tessendorf (eds), 857–868. Berlin: De Gruyter Mouton.Google Scholar
Pfeiffer, T., Hofmann, F., Hahn, F., Rieser, H. & Röpke, I.
2013 Gesture semantics reconstruction based on motion capturing and complex event processing: A circular shape example. In Proceedings of the SIGDIAL 2013 Conference, Maxine Eskenazi, Michael Strube, Barbara Di Eugenio & Jason D. Williams (eds), 270–279. Metz: ACL. https://​aclanthology​.info​/volumes​/proceedings​-of​-the​-sigdial​-2013​-conference
Russell, K., Wilkinson, E. & Janzen, T.
2011 ASL sign lowering as undershoot: A corpus study. Laboratory Phonology 2(2): 403–422.CrossrefGoogle Scholar
Sandler, W.
2012 Visual prosody. In Sign language, R. Pfau, M. Steinbach & B. Woll (eds), 55–76. Berlin: De Gruyter Mouton.CrossrefGoogle Scholar
Sandler, W. & Lillo-Martin, D.
2006 Sign Language and Linguistic Universals. Cambridge: CUP.CrossrefGoogle Scholar
Tang, G., Brentari, D., González, C., & Sze, F.
2010 Crosslinguistic variation in prosodic cues. In Sign languages, D. Brentari (ed), 519–542. Cambridge: CUP.CrossrefGoogle Scholar
Tyrone, M. E.
2007 Simultaneity in atypical signers – Implications for the structure of signed language. In Simultaneity in Signed Language: Form and Function [Current Issues in Linguistic Theory 281], M. Vermeerbergen, L. Leeson & O. Crasborn (eds), 317–335. Amsterdam: John Benjamins.CrossrefGoogle Scholar
Tyrone, M. E., Nam, H., Saltzman, E., Mathur, G. & Goldstein, L.
2010 Prosody and movement in American Sign Language: A task-dynamics approach. In Speech Prosody 2010 Conference Proceedings. http://​speechprosody2010​.illinois​.edu​/papers​/100957​.pdf
Tyrone, M. E., Atkinson, J. R., Marshall, J. & Woll, B.
2009 The effects of cerebellar ataxia on sign language production: A case study. Neurocase 15(5): 419–426.CrossrefGoogle Scholar
Tyrone, M. E. & Mauk, C. E.
2010 Sign lowering and phonetic reduction in American Sign Language. Journal of Phonetics 38(2): 317–328.CrossrefGoogle Scholar
Malaia, E., Wilbur, R. B.
2012 Kinematic signatures of telic and atelic events in ASL predicates. Language and Speech 55(3): 407–421. http://dx.doi.org/10.1177/0023830911422201Google Scholar
Wilbur, R. B.
1990 An experimental investigation of stressed sign production. International Journal of Sign Linguistics 1(1): 41–59.Google Scholar
1994 Eyeblinks and ASL phrase structure. Sign Language Studies 84: 221–240.CrossrefGoogle Scholar
1997 A prosodic/pragmatic explanation for word order variation in ASL with typological implications. In Lexical and Syntactic Constructions and the Construction of Meaning, Vol. 1 [Current Issues in Linguistic Theory 150], M. Verspoor, K. D. Lee & E. Sweetser (eds) 89–104. Amsterdam: John Benjamins.CrossrefGoogle Scholar
1999 Stress in ASL: Empirical evidence and linguistic issues. Language & Speech 42: 229–250.CrossrefGoogle Scholar
2000 Phonological and prosodic layering of non-manuals in American Sign Language. In The Signs of Language Revisited: Festschrift for Ursula Bellugi and Edward Klima, H. Lane & K. Emmorey (eds), 213–241. Hillsdale NJ: Lawrence Erlbaum Associates.Google Scholar
2009 Effects of varying rate of signing on ASL manual signs and non-manual markers. Language & Speech 52(2–3): 245–285.CrossrefGoogle Scholar
2010 The semantics-phonology interface. In Sign languages, D. Brentari (ed), 357–382. Cambridge: CUP.CrossrefGoogle Scholar
2011a Nonmanuals, semantic operators, domain marking, and the solution to two outstanding puzzles in ASL. Sign Language & Linguistics 14: 148–178.CrossrefGoogle Scholar
2011b Sign syllables. In The Blackwell Companion to Phonology, M. van Oostendorp, C. J. Ewen, E. Hume & K. Rice (eds), 1309–1334. Oxford: Blackwell.CrossrefGoogle Scholar
Wilbur, R. B. & Patschke, C.
1998 Body leans and marking contrast in ASL. Journal of Pragmatics 30: 275–303.CrossrefGoogle Scholar
Wilbur, R. B. & Malaia, E.
2008 Contributions of sign language research to gesture understanding: What can multimodal computational systems learn from sign language research. International Journal of Semantic Computing 2(1): 5–19.CrossrefGoogle Scholar
Wilbur, R. B. & Schick, B. S.
1987 The effects of linguistic stress on ASL signs. Language & Speech 30: 301–323.CrossrefGoogle Scholar
Wilbur, R. B.
2003 Representations of telicity in ASL. Chicago Linguistics Society 39, 354–368.Google Scholar
Cited by

Cited by 2 other publications

Catteau, Fanny
2021. Translating sign language poetry into spoken French. Sign Language & Linguistics Crossref logo
Krebs, Julia, Evie Malaia, Ronnie B. Wilbur & Dietmar Roehm
2020. Interaction between topic marking and subject preference strategy in sign language processing. Language, Cognition and Neuroscience 35:4 ► pp. 466 ff. Crossref logo

This list is based on CrossRef data as of 31 march 2022. Please note that it may not be complete. Sources presented here have been supplied by the respective publishers. Any errors therein should be reported to them.