Cover not available
Part of
Linking up with Video: Perspectives on interpreting practice and research
Edited by Heidi Salaets and Geert Brône
[Benjamins Translation Library 149] 2020
► pp. 203–233

Chapter 8
Eye-tracking in interpreter-mediated talk

From research to practice

A starting point for a multimodal analysis of interpreter-mediated interaction is the discussion on the pros and cons of new technologies especially for new forms of distant, remote or offsite interpreting. More specifically, in this contribution is argued that empirical multimodal analyses of participants’ and interpreters’ behavior in real-life interpreting settings may provide much-needed groundwork that helps to fuel this discussion. As one of the most central points of criticism raised by interpreting practitioners is that distant forms of interpreting lack certain characteristics that are typical for on-site or face-to-face interpreting (i.e. distant solutions are not true-to-life, Van Rotterdam & van den Hoogen 2011). It becomes increasingly important to pinpoint what these typical and natural characteristics are. Empirically grounded insights into these characteristics may then, in a research-technology-application loop, feed into innovative technologies, rendering novel interpreting solutions that are more true-to-life.
In this empirical analysis we focus on one specific phenomenon that has received increased attention in the recent literature in multimodal interaction analysis as well as interpreting studies, viz. eye gaze as a semiotic resource employed by speakers as well as hearers. The main questions here are how interpreters typically manage the turn-taking process in real-life interaction, among others by means of gaze, and how important visual access is to the co-participants for a smooth exchange. In order to answer the research questions, the researchers make use of eye-tracking technology to gain detailed information on interpreters’ and primary participants’ gaze behavior.
Article outline
  • 1.Face-to-face vs. remote interpreting
  • 2.Multimodal interaction: A brief technological state-of-the-art
  • 3.Interpreting: Multimodality & eye gaze
  • 4.Dataset & methodological considerations
  • 5.Empirical analysis
    • 5.1Multimodal feedback patterns in interpreter-mediated talk
      • a.Interpreter-directed feedback
      • b.Primary directed feedback
      • c.Dual feedback
    • 5.2Turn-taking in interpreter-mediated interactions
      • Adjacency pairs
      • Multiple turns
  • 6.Conclusion
  • Notes
  • References
  • Appendix. Transcription conventions
Published online: 13 January 2020
DOI logo
https://doi.org/10.1075/btl.149.09vra
References (65)
References
Auer, Peter. 2005. “Projection in interaction and projection in grammar.” Text 25 (1): 7–36. DOI logoGoogle Scholar
Bavelas, Janet., Linda Coates, and Trudy Johnson. 2002. “Listener responses as a collaborative process: The role of gaze.” Journal of Communication, 566–580. DOI logoGoogle Scholar
Bot, Hanneke. 2005. Dialogue Interpreting in Mental Health. Amsterdam and New York: Rodopi Publishers.Google Scholar
Braun Sabine. 2013. “Keep your distance? Remote interpreting in legal proceedings: A critical assessment of a growing practice.” Interpreting: international journal of research and practice in interpreting, 15 (2), pp.200–228. DOI logoGoogle Scholar
Braun, Sabine. 2015. “Remote interpreting.” In Routledge Encyclopedia of Interpreting Studies, ed. by Franz Pöchhacker, 346–348. London/New York: Routledge.Google Scholar
. 2017. “What a micro-analytical investigation of additions and expansions in remote interpreting can tell us about interpreters’ participation in a shared virtual space.” Journal of Pragmatics 107, 165–177. DOI logoGoogle Scholar
Braun, Sabine, and L. Judith Taylor,. 2012. Videoconference and Remote Interpreting in Criminal Proceedings. Cambridge/Antwerp: Intersentia.Google Scholar
Brenger, Bela, and Irene Mittelberg. (2015). “Shakes, nods and tilts. Motion-capture data profiles of speakers’ and listeners’ head gestures.” In: Proceedings of the 3rd Gesture and Speech in Interaction (GESPIN) Conference, Sept. 2015, Nantes. 43–48.Google Scholar
Brône, Geert and Bert Oben. 2015. “InSight Interaction. A multimodal and multifocal dialogue corpus.” Language Resources and Evaluation 49: 195–214. DOI logoGoogle Scholar
Brône, Geert, Bert Oben , Annelies Jehoul , Jelena Vranjes, and, Kurt Feyaerts. 2017. “Eye gaze and viewpoint in multimodal interaction management.” Cognitive Linguistics 28 (3): 449–484. DOI logoGoogle Scholar
Clayman, Steven E. 2013. “Turn-Constructional Units and the Transition-Relevance-Place.” In The Handbook of Conversation Analysis, ed. by Jack Sidnell, and Tanya Stivers, 150–166, Wiley-Blackwell.Google Scholar
Clark, Herbert. 1996. Using Language. Cambridge University press: Cambridge. DOI logoGoogle Scholar
Clark, Herbert, and Susan Brennan. 1991. “Grounding in communication.” In Perspectives on socially shared cognition, ed. by Lauren B. Resnick, John M. Levine, and Stephanie D. Teasley, Washington: APA Books. DOI logoGoogle Scholar
Davidson, Brad. 2002. “A model for the construction of conversational common ground in interpreted discourse.” Journal of Pragmatics 34: 1273–1300. DOI logoGoogle Scholar
. 2015. “Gaze.” In Routledge Encyclopedia of Interpreting Studies, ed. by Franz Pöchhacker, Nadja Grbic, Peter Mead, and Robin Setton, 168–168, New York: Routledge.Google Scholar
Davitti, Elena, and Sergio Pasquandrea. 2017. “Embodied participation: What multimodal analysis can tell us about interpreter-mediated encounters in pedagogical settings.” Elsevier Journal of Pragmatics 107: 105–128. DOI logoGoogle Scholar
Deppermann, Arnulf. 2013. “Turn-design at turn-beginnings: Multimodal resources to deal with tasks of turn-construction in German.” Journal of Pragmatics 46 (1): 91–121. DOI logoGoogle Scholar
Ellis, S. Ronald. 2004. “Videoconferencing in refugee hearings.” Ellis Report to the Immigration and Refugee Board Audit and Evaluation Committee. [URL].Google Scholar
Englund Dimitrova, Birgitta. 1997. “Degree of Interpreter Responsibility in the Interaction Process in Community Interpreting.” In The Critical Link: Interpreters in the Community, ed. by Silvana E. Carr, 147–164, Amsterdam/Philadelphia: Benjamins.Google Scholar
Esteban Causo, Jose. 2012. “Conference interpreting with information and communication technologies. Experiences from the European Commission DG Interpretation.” In Videoconference and remote interpreting in criminal proceedings, ed. by Sabine Braun, and Judith Taylor, 227–232 . Antwerp: Intersentia.Google Scholar
Ford, Cecilia, Barbara Fox, and A. Sandra Thompson. 1996. “Practices in the construction of turns: The TCU revisted.” Pragmatics 6 ( 3 ): 427–454 . DOI logoGoogle Scholar
Gardner, Rod. 2001. When Listeners Talk: Response tokens and listener stance. Amsterdam: J. Benjamins Publishing. DOI logoGoogle Scholar
Gerwing, Jennfer, & Bavelas Janet (2013). “The social interactive nature of gestures: Theory, assumptions, methods, and findings. In Body-language-communication, Volume I, Contemporary approaches, ed. by Cornelia Müller, Allan Cienki, Ellen Fricke, Silva Ladewig, David McNeil & Sedinha Tessendorf. 816-831. Berlin: Mouton De Gruyter.Google Scholar
Goffman, Erving. 1981. Forms of talk. Oxford: Basil Blackwell.Google Scholar
Goodwin, Charles. 1981. Conversational Organization. Interaction between Speakers and Hearers. New York, London: Academic.Google Scholar
Hayashi, Makoto. 2013. “Turn Allocation and Turn Sharing.” In The Handbook of Conversation Analysis, ed. by Jack Sidnell and Tanya Stivers, 167–190, Wiley-Blackwell.Google Scholar
Heath, Christian. 1986. Body Movement and Speech in Medical Interaction. Cambridge, U.K.: Cambridge University Press. DOI logoGoogle Scholar
Holler, Judith, Kobin H.,Kendrick, Marisa Casillas, and Stephen, C. Levinson . 2016. Turn-Taking in Human Communicative Interaction. Lausanne: Frontiers Media. DOI logo.Google Scholar
Holler, Judith, and Kobin. H. Kendrick. 2015.“Unaddressed participants’ gaze in multi-person interaction: Optimizing recipiency.” Frontiers in Psychology 6: 98. DOI logoGoogle Scholar
Jokinen, Kristiina. 2010. “Non-verbal signals for turn-taking & feedback.” Proc. of 7th Int. Conf. on Language Resources & Evaluation (LREC) International Universal Communication Symposium.Google Scholar
Kendon, Adam. 1967. “Some functions of gaze-direction in social interaction.” Acta Psychologica 26: 22–63. DOI logoGoogle Scholar
Kipp, Michael. 2014. “ANVIL: A Universal Video Research Tool.” In Handbook of Corpus Phonology, ed. by Jacques Durand, Ulrike Gut, and Gjert Kristofferson, 420–436. Oxford: Oxford University Press.Google Scholar
Krystallidou, Demi. 2016. “Investigating the Interpreter’s Role(s): The A.R.T. Framework.” Interpreting 18 (2): 172–197. DOI logoGoogle Scholar
Lang, Ranier. 1978. “Behavioral aspects of liaison interpreters in Papua New Guinea: some preliminary observations.” In Language Interpretation and Communication, ed. by David Gerver, and H. Wallace Sinaiko, 231–244. New Yotk/London: Plenum Press. DOI logoGoogle Scholar
Lerner, Gene. H. 2003. “Selecting next speaker: The context-sensitive operation of a context-free organization.” Language in Society 32: 177–201. DOI logoGoogle Scholar
Llewellyn-Jones, Peter, and Robert. G. Lee, 2014. Redefining the Role of the Community Interpreter: The Concept of role-space. Lincoln: SLI Press.Google Scholar
Li, Shuangyu. 2015. “Nine Types of Turn-taking in Interpreter-mediated GP Consultations.” Applied Linguistics Review 6 (1): 73–96. DOI logoGoogle Scholar
Licoppe, Christian, and Clair-Antoine Veyrier,. 2017. “How to show the interpreter on screen? The normatice organization of visual ecologies in multilingual courtrooms with video links.” Elsevier Journal of Pragmatics 107: 147–164. DOI logoGoogle Scholar
Linell, Per, Linda Jönsson, and Cecilia Wadensjö. 1992. “Establishing Communicative Contact through a Dialogue Interpreter.” In LSP Communication/Fachsprachliche Kommunikation, ed. by Annette Grindsted, and Johannes Wagner, 125–142, Tübingen: Narr Verlag.Google Scholar
Mason, Ian. 2012. “Gaze, positioning and identity in interpreter-mediated dialogues.” In Coordinating Participation in Dialogue Interpreting, ed. by Claudio. Baraladi and Laura Gavioli, 177–199, Amsterdam/Philadelphia: Benjamins Publishing. DOI logoGoogle Scholar
Merlini, Raffaela, and Roberta Favaron. 2005. “Examining the “voice of interpreting” in speech pathology.” Interpreting 7 (2): 263–302. DOI logoGoogle Scholar
Mondada, Lorenza. 2007. “Multimodal resources for turn-taking: pointing and emergence of next speakers.” Discourse Studies: 194–225. DOI logoGoogle Scholar
. 2013. “Video as a tool in the social sciences.” In Body – Language – Communication: An International Handbook on Multimodality in Human Interaction , Vol. 1, ed. by Cornelia Müller, Alan Cienki, Ellen Fricke, Silva H. Ladewig, David McNeill, and Sedinha Teßendorf, 982–992. Berlin: De Gruyter Mouton.Google Scholar
. 2016. “Challenges of multimodality: Language and the body in social interaction.” Journal of Sociolinguistics 20–3, 336–366. DOI logoGoogle Scholar
Moser-Mercer, Barbara. 2003. “Remote interpreting: assessment of human factors and performance parameters’” Communicate! Summer 2003. Available at [URL].Google Scholar
. 2005. “Remote interpreting: issues of multi-sensory integration in a multilingual task,” Meta, 50 (2), 727–738. DOI logoGoogle Scholar
Oben, Bert, and Geert Brône. 2015. “What you see is what you do: On the relation between gaze and gesture in multimodal alignment.” Language and Cognition 7 (4): 546–562. DOI logoGoogle Scholar
Pasquandrea, Sergio. 2011. “Managing multiple actions through multimodality: Doctors’ involvement in interpreter-mediated interactions.” Language in Society 40 (4): 455–481. DOI logoGoogle Scholar
Pitsch Karola, Bernhard-Andreas Brüning, Christian Schnier, Holger Dierker, and Sven Wachsmuth. 2010. Linking conversation analysis and motion capturing: How to robustly track multiple participants. In Proceedings of the LREC Workshop on Multimodal Corpora: Advances in Capturing, Coding and Analyzing Multimodality (MMC 2010), ed. by : Michael Kipp, Jean-Claude Martin, Patrizia Paggio, and Dirk Heylen, 63–69.Google Scholar
Price, Erika, Eliseo Pérez-Stable, Dana Nickleach, Monica López, and Leah Karliner. 2012. “Interpreter perspectives of in-person, telephonic, and videoconferencing medical interpretation in clinical encounters.” Patient Education & Counseling, 87 (2), 226–232. DOI logoGoogle Scholar
Rossano, Federico. 2013. “Gaze in Conversation.” In The Handbook of Conversation Analysis, ed. by Jack Sidnell, and Tanya Stivers, 308–329. Oxford: Wiley-Blackwell.Google Scholar
Roziner, Ilan, and Miriam Shlesinger. 2010. “Much ado about something remote: Stress and performance in remote interpreting.” Interpreting 12 (2): 214–247. DOI logoGoogle Scholar
Sacks, Harvey, Emanuel A. Schegloff, and Gail Jefferson. 1974. “A simplest systematics for the organization of turn-taking in conversation.” Language 50: 696–735. DOI logoGoogle Scholar
Schmitt, Reinhold. 2005.“Zur multimodalen Struktur von turn-taking.” Gesprächsforschung 6: 17–61.Google Scholar
Selting, Margaret. et al. 2009.“Gesprächsanalytisches Transkriptionssystem 2 (GAT 2).” Gesprächsforschung – Online-Zeitschrift zur verbalen Interaktion 10: 353–402. [[URL]]Google Scholar
Sloetjes, Han, and Peter Wittenburg 2008. “Annotation by category – ELAN and ISO DCR.” In: Proceedings of the 6th International Conference on Language Resources and Evaluation (LREC 2008).Google Scholar
Stivers, Tania and Federico Rossano. 2010. “Mobilizing Response.” Research on Language and Social Interaction 43 (1): 3–31. DOI logoGoogle Scholar
Vandemoortele, Sarah, Stijn De Beugher, Geert Brône, Kurt Feyaerts, Toon Goedemé, Thomas De Baets, and Stijn Vervliet. 2016. Into the Wild. Musical Communication in Ensemble Playing. Leuven, Acco.Google Scholar
Van Rotterdam, Peter, and Ronald van den Hoogen. 2011. “True-to-life requirements for using videoconferencing in legal proceedings.” In Videoconference and remote interpreting in criminal proceedings, ed. by Sabine Braun, and Judith L. Taylor, 187–197. Guildford: University of Surrey.Google Scholar
Vranjes, Jelena, Geert Brône, and Kurt Feyaerts. 2018. “Dual feedback in interpreter-mediated interactions: on the role of gaze in the production of listener responses.” Journal of Pragmatics 134: 15–30. DOI logoGoogle Scholar
Vranjes, Jelena, Hanneke Bot, Kurt Feyaerts, and Geert Brône. (2019). “Displaying recipiency in an interpreter-mediated dialogue: an eye-tracking study. In Eye-tracking in interaction. Studies on the role of eye gaze in dialogue, ed. by Bert Oben and G. Brône . Amsterdam: Benjamins Publishing.Google Scholar
Vranjes, Jelena, Hanneke Bot , Kurt Feyaerts, and Geert Brône (2019). “Affiliation in interpreter-mediated therapeutic talk: on the relationship between gaze and head nods.” Interpreting. DOI logoGoogle Scholar
Vranjes, Jelena. 2017. “Dual feedback and the management of intersubjectivity in interpreter-mediated talk. Mapping Multimodal Dialogue (MaMuD 4).” Frankfurt/Oder, Germany, 23–24 November 2017.Google Scholar
Wadensjö, Cecilia. 1998. Interpreting as interaction. London and New York: Longman.Google Scholar
Cited by (5)

Cited by five other publications

Jerkovic, Tiana
2024. Space, body and presence. Interpreting. International Journal of Research and Practice in Interpreting 26:2 ► pp. 201 ff. DOI logo
Argaman, Einav
2023. Neither speaker nor recipient: The middle-distance look of unaddressed participants. Discourse Studies 25:5 ► pp. 573 ff. DOI logo
Salaets, Heidi & Katalin Balogh
2023. Are interpreters and interpreting technology ready for the post-Covid era?. In Introducing New Hypertexts on Interpreting (Studies) [Benjamins Translation Library, 160], ► pp. 254 ff. DOI logo
Zhang, Xiaojun, Gloria Corpas Pastor & Jing Zhang
2023. Videoconference interpreting goes multimodal. In Interpreting Technologies – Current and Future Trends [IVITRA Research in Linguistics and Literature, 37], ► pp. 169 ff. DOI logo
Doherty, Stephen, Natalie Martschuk, Jane Goodman-Delahunty & Sandra Hale
2022. An Eye-Movement Analysis of Overt Visual Attention During Consecutive and Simultaneous Interpreting Modes in a Remotely Interpreted Investigative Interview. Frontiers in Psychology 13 DOI logo

This list is based on CrossRef data as of 31 december 2024. Please note that it may not be complete. Sources presented here have been supplied by the respective publishers. Any errors therein should be reported to them.