Cover not available
Article published In:
The Semiotic Diversity of Language: The Case of Signed Languages
Edited by Alysson Lepeut and Inez Beukeleers
[Belgian Journal of Linguistics 36] 2022
► pp. 179–211

Gesture at the crossroads

Interpreting semiotic fields in a task-based exchange

 | Gallaudet University
Task-based exchanges are known to have multiple co-occurring interactive structures that prompt interlocutors to integrate various semiotic fields (Goodwin 2000). In addition to underlying norms of cooperative interaction, task-based exchanges include joint accomplishment of some activity. In this study, a group of co-workers (two hearing and one deaf) engaged in a team-building exercise where they jointly constructed a container fit for protecting an egg. They communicated, in part, via a sign language interpreter. Clark’s (2016) methods of communication were applied to the interpreter-mediated interaction (IMI) to determine whether certain semiotic modes were more (or less) accessible to the respective linguistic groups. Micro-analysis of three activity phases revealed moments when the participants responded to and integrated semiotic fields displayed in front of them. While not all content was accessible, like the complex depictive structures produced by the deaf person, some instances of mutual understanding without interpretation were achieved via integration of meaningful components of visibly accessible semiotic fields. Results challenge traditional views of IMIs where interpreters are positioned as channelling communicative content exclusively through them.
Keywords: interpreter-mediated interaction, depiction, semiotics, conversation analysis, multimodality, American Sign Language/English
Article outline
  • 1.Introduction
  • 2.Background
    • 2.1(Multimodal) Language in interaction
    • 2.2Analyzing interactions with interpreters
  • 3.Methodology
  • 4.Analysis
    • 4.1Interaction sequence 1: Instruction-reading
    • 4.2Interaction sequence 2: Idea-proposing
    • 4.3Interaction sequence 3: Idea-implementing
  • 5.Discussion
  • 6.Conclusion
  • Acknowledgements
  • Notes
  • References
Published online: 9 June 2023
DOI logo
https://doi.org/10.1075/bjl.00075.sha
References (41)
References
Baraldi, Claudio. 2012. “Interpreting as Dialogic Mediation: The Relevance of Expansions.” In Coordinating Participation in Dialogue Interpreting, ed. by Claudio Baraldi, and Laura Gavioli, 297–326. Amsterdam/Philadelphia: John Benjamins. DOI logoGoogle Scholar
Bavelas, Janet. 1994. “Gestures as Part of Speech: Methodological Implications.” Research on Language & Social Interaction 27(3): 201–221. DOI logoGoogle Scholar
Berge, Sigrid S., and Gøril Thomassen. 2016. “Visual Access in Interpreter-Mediated Learning Situations for Deaf and Hard-of-Hearing High-School Students where an Artifact is in Use. Journal of Deaf Studies and Deaf Education 21(2): 187–199. DOI logoGoogle Scholar
Beukeleers, Inez, and Myriam Vermeerbergen. 2022. “Show Me What You’ve B/Seen: A Brief History of Depiction.” Frontiers in Psychology 131: 808814. DOI logoGoogle Scholar
Brown, Penelope, and Stephen Levinson. 1987. Politeness: Some Universals in Language Usage. Cambridge: Cambridge University Press. DOI logoGoogle Scholar
Chui, Kawai. 2009. “Conversational Coherence and Gesture.” Discourse Studies 111: 661–680. DOI logoGoogle Scholar
Clark, Herbert H. 2016. “Depicting as a Method of Communication.” Psychological Review 123 (3): 324–347. DOI logoGoogle Scholar
2019. “Depicting in Communication.” In Human Language: From Genes and Brains to Behavior, ed. by Peter Hagoort, 235–247. Cambridge, MA: MIT Press. DOI logoGoogle Scholar
Degand, Liesbeth, and Anne-Catherine Simon. 2009. “On Identifying Basic Discourse Units in Speech: Theoretical and Empirical Issues.” Discours [On-line], 4|2009. URL: [URL]
Dickinson, Jules. 2017. Sign Language Interpreting in the Workplace. Washington, DC: Gallaudet University Press. DOI logoGoogle Scholar
Dudis, Paul. 2011. “The Body in Scene Depictions.” In Discourse in Signed Languages, ed. by Cynthia Roy, 3–45. Washington, DC: Gallaudet University Press. DOI logoGoogle Scholar
Dudis, Paul, and Emily Shaw. 2022. An Investigation of Theme Buoys in ASL. Poster presented at the 9th Conference of the International Society for Gesture Studies, Chicago, IL, July 13–15, 2022.
Enfield, Nick J. 2009. The Anatomy of Meaning: Speech, Gesture, and Composite Utterances. Cambridge: Cambridge University Press. DOI logoGoogle Scholar
Ferrara, Lindsay. 2020. “Some Interactional Functions of Finger Pointing in Signed Language Conversations.” Glossa: A Journal of General Linguistics 51: 88. DOI logoGoogle Scholar
Ferrara, Lindsay, and Gabrielle Hodge. 2018. “Language as Description, Indication, and Depiction.” Frontiers in Psychology 91: 716. DOI logoGoogle Scholar
Gerwing, Jennifer, and Meredith Allison. 2009. “The Relationship between Verbal and Gestural Contributions in Conversation: A Comparison of Three Methods.” Gesture 9 (3): 312–336. DOI logoGoogle Scholar
Gerwing, Jennifer, and Janet Bavelas. 2004. “Linguistic Influences on Gesture’s Form.” Gesture 4 (2): 157–195. DOI logoGoogle Scholar
Girard-Groeber, Simone. 2015. “The Management of Turn Transition in Signed Interaction through the Lens of Overlaps.” Frontiers in Psychology 61: 741. DOI logoGoogle Scholar
Goffman, Erving. 1981. Forms of Talk. Philadelphia, PA: University of Pennsylvania Press.Google Scholar
Goodwin, Charles. 1986. “Gestures as a Resource for the Organization of Mutual Orientation.” Semiotica 62 (1–2): 29–49. DOI logoGoogle Scholar
. 2000. “Action and Embodiment within Situated Human Interaction.” Journal of Pragmatics 321: 1489–1522. DOI logoGoogle Scholar
. 2013. “The Co-Operative, Transformative Organization of Human Action and Knowledge.” Journal of Pragmatics 461: 8–23. DOI logoGoogle Scholar
Kendon, Adam. 2004. Gesture: Visible Action as Utterance. Cambridge: Cambridge University Press. DOI logoGoogle Scholar
Kusters, Annelies. 2017. “Gesture-Based Customer Interactions: Deaf and Hearing Mumbaikars? Multimodal and Metrolingual practices.” International Journal of Multilingualism 141: 283–302. DOI logoGoogle Scholar
Lepeut, Alysson, and Emily Shaw. 2022. “Time is Ripe to Make Interactional Moves: Bringing Evidence from Four Languages across Modalities.” Frontiers in Communication 71: 780124. DOI logoGoogle Scholar
Liddell, Scott. 2003. Grammar, Gesture, and Meaning in American Sign Language. Cambridge: Cambridge University Press. DOI logoGoogle Scholar
Metzger, Melanie. 1999. Sign Language Interpreting: Deconstructing the Myth of Neutrality. Washington, DC: Gallaudet University Press.Google Scholar
Müller, Cornelia. 2004. “Forms and Uses of the Palm Up Open Hand: A Case of a Gesture Family?” In The Semantics and Pragmatics of Everyday Gestures, ed. by Corneila Müller, and Roland Posner, 233–256. Berlin: Weidler.Google Scholar
Peirce, Charles S. 1955 (1893). “Logic as Semiotic: The Theory of Signs.” In Philosophical Writings of Peirce, ed. by Justus Buchler, 98–119. New York: Dover Publications.Google Scholar
Roy, Cynthia. 2000. Interpreting as a Discourse Process. New York: Oxford University Press.Google Scholar
Sacks, Harvey, Schegloff, Emanuel, and Gail Jefferson. 1974. “A Simplest Systematics for the Organization of Turn-Taking for Conversation.” Language 50 (4): 696–735. DOI logoGoogle Scholar
Schegloff, Emanuel, and Harvey Sacks. 2002. “Home Position.” Gesture 2 (2): 133–146. DOI logoGoogle Scholar
Schiffrin, Deborah. 1987. Discourse Markers. Cambridge: Cambridge University Press. DOI logoGoogle Scholar
Sequeiros, Xosé Rosales. 1998. “Interlingual Impoverishment in Translation.” Bulletin of Hispanic Studies 75 (1): 145–157. DOI logoGoogle Scholar
Shaw, Emily. 2019. Gesture in Multiparty Interaction. Washington, DC: Gallaudet University Press.Google Scholar
Streeck, Jürgen. 2008. “Depicting by Gesture.” Gesture 8 (3): 285–301. DOI logoGoogle Scholar
. 2013. “Interaction and the Living Body.” Journal of Pragmatics 461: 69–90. DOI logoGoogle Scholar
Taub, Sarah. 2001. Language from the Body: Iconicity and Metaphor in American Sign Language. Cambridge: Cambridge University Press. DOI logoGoogle Scholar
Ten Have, Paul. 2007. Doing Conversation Analysis (2nd ed.). Los Angeles, CA: SAGE Publications. DOI logoGoogle Scholar
Van Herreweghe, Mieke. 2002. “Turn-taking Mechanisms and Active Participation in Meetings with Deaf and Hearing Participants in Flanders.” In Turn-taking, Fingerspelling, and Contact in Signed Languages, ed. by Ceil Lucas, 73–103. Washington, DC: Gallaudet University Press.Google Scholar
Wadensjö, Cecilia. 1998. Interpreting as Interaction. New York, NY: Routledge.Google Scholar