A new technique for analyzing narrative prosodic effects in sign languages using motion capture technology
The present paper addresses prosody at a sentence level analysis within short narratives, developing a novel method based on a combination of data. Our first objective, then, is to establish the validity of our new method by confirming the findings of previous reports on Phrase Final Lengthening. Our second objective is to further extend the validity of the new method by confirming a prior claim regarding the role of increased velocity for signs that are stressed in sentences. Finally, we report new results in the form of patterns across signs within sentences using our new method.
Article outline
- 1.Introduction
- 1.1Perceptual studies of prosody and fluency
- 1.2Production studies of fluency and prosody
- 1.3Prior kinematic work
- 1.3.1Prior motion capture work showing sign lowering
- 1.3.2Prior motion capture work showing verb class differences
- 1.3.3Prior motion capture work showing phrase final lengthening
- 1.3.4Prior motion capture work showing stress marking
- 2.The present study
- 2.1Procedure
- 2.1.1Motion capture method for current report
- 2.1.2Coding for non-kinematic linguistic and perceptual variables
- 2.1.3Coding for kinematic variables
- 2.2Analyses
- 2.3Results
- 2.3.1Kinematic analysis by sentence position
- 2.3.2Kinematic analysis of stress
- 2.3.3More general test of relationships among measured variables and weight
- 2.4Summary
- 3.Conclusion
-
Acknowledgements
-
Notes
-
References
References (56)
Allen, G. D., Wilbur, R. B. & Schick, B.
1991 Aspects of rhythm in American Sign Language.
Sign Language Studies 72: 297–320.
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Boyes Braem, P.
1999 Rhythmic temporal patterns in the signing of deaf early and later learners of Swiss German Sign Language.
Language & Speech 42: 177–208.
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Brentari, D.
1998 A Prosodic Model of Sign Language Phonology. Cambridge MA: The MIT Press.
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Brentari, D. & Crossley, L.
Brentari, D., Nadolske, M. & Wolford, G.
2012 Can experience with co-speech gesture influence the prosody of a sign language? Sign language prosodic cues in bimodal bilinguals.
Bilingualism: Language and Cognition 15(2): 402–412.
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Brentari, D., González, C., Seidl, A. & Wilbur, R. B.
2011 Sensitivity to visual prosodic cues in signers and nonsigners.
Language & Speech 54: 49–72.
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
González, C.
2011 Perception of Prosody in American Sign Language. PhD dissertation, Purdue University.
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Green, K.
1984 Sign boundaries in American Sign Language.
Sign Language Studies 42: 65–91.
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Grosjean, F. & Lane, H.
1977 Pauses and syntax in American Sign Language.
Cognition 5: 101–117.
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Grosvald, M. A.
2009 Long-distance Coarticulation: A Production and Perception Study of English and American Sign Language. PhD dissertation, University of California at Davis.
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Kantor, R.
1978 Identifying native and second-language signers. In Sign Language Research,
R. B. Wilbur (ed),
Special issue of Communication and Cognition 11(1): 39–55.
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Kröger, B. J., Birkholz, P., Kannampuzha, J., Kaufmann, E. & Mittelberg, I.
2011 Movements and holds in fluent sentence production of American Sign Language: The action-based approach.
Cognitive Computation 3(3): 449–465.
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Liddell, S. K.
1977 An Investigation into the Syntactic Structure of American Sign Language. PhD dissertation, University of California, San Diego.
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Lupton, L. K.
1998 Fluency in American Sign Language.
Journal of Deaf Studies and Deaf Education 3(4): 320–328.
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Lupton, L. K. & Zelaznik, H.
1990 Motor learning in sign language students.
Sign Language Studies 67: 153–174.
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Weast, T.
2008 Questions in American Sign Language: A Quantitative Analysis of Raised and Lowered Eyebrows. Phd dissertation, University of Texas, Arlington, U.S.A.
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Malaia, E.
2014 It still isn’t over: Event boundaries in language and perception.
Language and Linguistics Compass 8(3): 89–98.
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Malaia, E.
2017 Current and future methodologies for quantitative analysis of information transfer in sign language and gesture data.
Behavioral and Brain Sciences 40.
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Malaia, E., Borneman, J. D. & Wilbur, R. B.
2016 Assessment of information content in visual signal: analysis of optical flow fractal complexity.
Visual Cognition 24(3): 246–251.
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Malaia, E., Borneman, J. D. & Wilbur, R. B.
2017 Information transfer capacity of articulators in American Sign Language.
Language and Speech. doi:
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
.
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Malaia, E., Ranaweera, R., Wilbur R. B. & Talavage, T. M.
2012 Neural representation of event structure in American Sign Language: fMRI comparison of cortical activations in deaf signers and hearing non-signers.
Neuroimage 59: 4094–4101.
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Malaia, E. & Wilbur, R. B.
2012a Kinematic signatures of telic and atelic events in ASL predicates.
Language & Speech 55(3): 407–421.
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Malaia, E. & Wilbur, R. B.
2012b Telicity expression in the visual modality. In
Telicity, Change, and State: A Cross-categorial View of Event Structure,
V. Demonte &
L. McNally (eds), 122–136. Oxford: OUP.
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Malaia, E., Wilbur, R. B. & Milković, M.
2013 Kinematic parameters of signed verbs at the morpho-phonology interface.
Journal of Speech, Language and Hearing Research 56: 1677–1688.
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Malaia, E., Wilbur, R. B. & Weber-Fox, C.
2013 Event end-point primes the undergoer argument: Neurobiological bases of event structure processing. In
Studies in the Composition and Decomposition of Event Predicates,
B. Arsenijevic,
B. Gehrke &
R. Marin (eds), 231–248. Dordrecht: Springer.
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Mauk, C. E.
2003 Undershoot in Two Modalities: Evidence from Fast Speech and Fast Signing. PhD dissertation, University of Texas at Austin.
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Mauk, C. E. & Tyrone, M. E.
2008 Sign lowering as phonetic reduction in American Sign Language. In
Proceedings of the 2008 International Seminar on Speech Production,
R. Sock,
S. Fuchs &
Y. Laprie (eds), 185–188.
[URL]![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Mauk, C. E. & Tyrone, M. E.
McDonald, J., Wolfe, R., Wilbur, R. B., Moncrief, R., Malaia, E., Fujimoto, S., Baowidan, S. & Stec, J.
2016 A new tool to facilitate prosodic analysis of motion capture data and a data-driven technique for the improvement of avatar motion. In
Proceedings of the Seventh Workshop on the Representation and Processing of Sign Languages: Corpus Mining (LREC-2016), 153–158, Portorož, Slovenia.
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Milković, M.
2011 Verb Classes in Croatian Sign Language (HZJ): Syntactic and Semantic Properties PhD dissertation, University of Zagreb, Croatia.
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Nespor, M. & Sandler, W.
1999 Prosody in Israeli Sign Language.
Language and Speech 42: 143–176.
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Ormel, E. & Crasborn, O.
2012 Prosodic correlates of sentences in signed languages: A literature review and suggestions for new types of studies.
Sign Language Studies 12(2): 279–315.
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Pfeiffer, T.
2013 Documentation of gestures with motion capture. In
Body – Language – Communication,
C. Müller,
A. Cienki,
E. Fricke,
H. S. Ladewig,
D. McNeill &
S. Tessendorf (eds), 857–868. Berlin: De Gruyter Mouton.
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Pfeiffer, T., Hofmann, F., Hahn, F., Rieser, H. & Röpke, I.
2013 Gesture semantics reconstruction based on motion capturing and complex event processing: A circular shape example. In Proceedings of the SIGDIAL 2013 Conference,
Maxine Eskenazi,
Michael Strube,
Barbara Di Eugenio &
Jason D. Williams (eds), 270–279. Metz: ACL.
[URL]
Russell, K., Wilkinson, E. & Janzen, T.
2011 ASL sign lowering as undershoot: A corpus study.
Laboratory Phonology 2(2): 403–422.
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Sandler, W.
2012 Visual prosody. In
Sign language,
R. Pfau,
M. Steinbach &
B. Woll (eds), 55–76. Berlin: De Gruyter Mouton.
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Sandler, W. & Lillo-Martin, D.
2006 Sign Language and Linguistic Universals. Cambridge: CUP.
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Tang, G., Brentari, D., González, C., & Sze, F.
2010 Crosslinguistic variation in prosodic cues. In
Sign languages,
D. Brentari (ed), 519–542. Cambridge: CUP.
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Tyrone, M. E., Nam, H., Saltzman, E., Mathur, G. & Goldstein, L.
2010 Prosody and movement in American Sign Language: A task-dynamics approach. In
Speech Prosody 2010 Conference Proceedings.
[URL]
Tyrone, M. E., Atkinson, J. R., Marshall, J. & Woll, B.
2009 The effects of cerebellar ataxia on sign language production: A case study.
Neurocase 15(5): 419–426.
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Tyrone, M. E. & Mauk, C. E.
2010 Sign lowering and phonetic reduction in American Sign Language.
Journal of Phonetics 38(2): 317–328.
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Malaia, E., Wilbur, R. B.
Wilbur, R. B.
1990 An experimental investigation of stressed sign production.
International Journal of Sign Linguistics 1(1): 41–59.
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Wilbur, R. B.
1994 Eyeblinks and ASL phrase structure.
Sign Language Studies 84: 221–240.
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Wilbur, R. B.
1999 Stress in ASL: Empirical evidence and linguistic issues.
Language & Speech 42: 229–250.
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Wilbur, R. B.
2000 Phonological and prosodic layering of non-manuals in American Sign Language. In
The Signs of Language Revisited: Festschrift for Ursula Bellugi and Edward Klima,
H. Lane &
K. Emmorey (eds), 213–241. Hillsdale NJ: Lawrence Erlbaum Associates.
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Wilbur, R. B.
2009 Effects of varying rate of signing on ASL manual signs and non-manual markers.
Language & Speech 52(2–3): 245–285.
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Wilbur, R. B.
2010 The semantics-phonology interface. In
Sign languages,
D. Brentari (ed), 357–382. Cambridge: CUP.
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Wilbur, R. B.
2011b Sign syllables. In
The Blackwell Companion to Phonology,
M. van Oostendorp,
C. J. Ewen,
E. Hume &
K. Rice (eds), 1309–1334. Oxford: Blackwell.
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Wilbur, R. B. & Patschke, C.
1998 Body leans and marking contrast in ASL.
Journal of Pragmatics 30: 275–303.
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Wilbur, R. B. & Malaia, E.
2008 Contributions of sign language research to gesture understanding: What can multimodal computational systems learn from sign language research.
International Journal of Semantic Computing 2(1): 5–19.
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Wilbur, R. B. & Schick, B. S.
1987 The effects of linguistic stress on ASL signs.
Language & Speech 30: 301–323.
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Wilbur, R. B.
2003 Representations of telicity in ASL.
Chicago Linguistics Society 39, 354–368.
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Cited by (3)
Cited by 3 other publications
Napoli, Donna Jo & Rachel Sutton-Spence
Krebs, Julia, Evie Malaia, Ronnie B. Wilbur & Dietmar Roehm
2020.
Interaction between topic marking and subject preference strategy in sign language processing.
Language, Cognition and Neuroscience 35:4
► pp. 466 ff.
![DOI logo](//benjamins.com/logos/doi-logo.svg)
This list is based on CrossRef data as of 26 june 2024. Please note that it may not be complete. Sources presented here have been supplied by the respective publishers.
Any errors therein should be reported to them.