Cover not available
Part of
Mobile Eyetracking: New avenues for the study of gaze in social interaction
Edited by Elisabeth Zima and Anja Stukenbrock
[Pragmatics & Beyond New Series 351] 2025
► pp. 1–21

Chapter 1
Introduction

Mobile eye tracking for the study of gaze in social interaction

 | University of Heidelberg
 | University of Freiburg
Article outline
  • 1.Gaze in social interaction: A multifunctional resource
  • 2.The advent of mobile eye tracking: A turning point in Conversation Analysis?
  • 3.The chapters of this volume
  • Author queries
  • Acknowledgements
  • Note
  • References
This content is being prepared for publication; it may be subject to changes.
References (96)
References
Argyle, Michael and Cook, Mark. 1976. Gaze and Eye contact. London: Cambridge University Press.Google Scholar
Attardo, Salvatore and Pickering, Lucy. 2022. Eye Tracking in Linguistics. London: Bloomsbury Academic.Google Scholar
Auer, Peter. 2018. “Gaze, Addressee Selection and Turn-Taking in Three-Party Interaction.” In Eye-Tracking in Interaction: Studies on the Role of Eye Gaze in Dialogue, ed. by Geert Brône and Bert Oben. 197–231. Amsterdam, Philadelphia: John Benjamins. DOI logoGoogle Scholar
. 2021a. “Turn-allocation and gaze: A multimodal revision of the “current-speaker-selects-next” rule of the turn-taking system of conversation analysis.” Discourse Studies 23(2), 117–140. DOI logoGoogle Scholar
Auer, Peter & Zima, Elisabeth. 2021. “On word searches, gaze, and co-participation.” Gesprächsforschung — Online — Zeitschrift zur verbalen Interaktion 22, 390–425.Google Scholar
Auer, Peter, Laner, Barbara, Pfeiffer, Martin, Botsch, Kerstin. 2024. “A multimodal investigation of the format ‘perception imperative + exclamative’ based on mobile eye-tracking data.” In New Perspectives in Interactional Linguistic Research. ed. by Margret Selting and Dagmar Barth-Weingarten. 242–271. Amsterdam: Benjamins. DOI logoGoogle Scholar
Balantani, Angeliki. 2022. “Transitioning between activities in joint projects: The case of German so.“ Gesprächsforschung — Online-Zeitschrift zur verbalen Interaktion 23, 343–370.Google Scholar
Balantani, Angeliki and Lázaro, Stefani. 2021. “Joint attention and reference construction: The role of pointing and “so”.” Language and Communication 79(6), 33–52. DOI logoGoogle Scholar
Bavelas, Janet, Coates, Linda and Johnson, Trudy. 2002. “Listener responses as a collaborative process: The role of gaze.” Journal of Communication 52, 566–580. DOI logoGoogle Scholar
Beattie, Geoffrey W. 1978. “Floor apportionment and gaze in conversational dyads”. British Journal of Social & Clinical Psychology 17(1), 7–15. DOI logoGoogle Scholar
1979. “Planning units in spontaenous speech: Some evidence from hesitations in speech and speaker gaze direction in conversation.” Linguistics 17, 61–78. DOI logoGoogle Scholar
Brône, Geert and Oben, Bert. 2015. “InSight Interaction: A multimodal and multifocal dialogue corpus.” Language Resources and Evaluation 49(1), 195–214. DOI logoGoogle Scholar
Brône, Geeert and Oben, Bert Eds. 2018. Eye-tracking in Interaction: Studies on the role of eye gaze in dialogue. Amsterdam: John Benjamins. DOI logoGoogle Scholar
Brooks, Rechele and Meltzoff, Andrew N. 2014. “Gaze following: A mechanism for building social connections between infants and adults.” In Mechanisms of social connection: From brain to group. ed. by Mario Mikulincer and Phillip R. Shaver. 167–183. American Psychological Association. DOI logoGoogle Scholar
Carpenter, Malinda, Katherine Nagell, Michael Tomasello, George Butterworth, and Chris Moore. 1998. “Social Cognition, Joint Attention, and Communicative Competence from 9 to 15 Months of Age.” Monographs of the Society for Research in Child Development 63(4): 1–166. DOI logoGoogle Scholar
Clark, Herbet H. 1996. Using language. Cambridge: Cambridge University Press. DOI logoGoogle Scholar
2021. “Common ground.” In International Encyclopedia of Linguistic Anthropology, ed. by James Stanlaw. New York: Wiley. [URL]
Degutyte, Ziedune and Astell, Arlene. 2021. “The Role of Eye Gaze in Regulating Turn Taking in Conversations: A Systematized Review of Methods and Findings.” Frontiers in Psychology 12: 616471. DOI logoGoogle Scholar
Deppermann, Arnulf. 2015. “Pragmatik revisited”. In Sprachwissenschaft im Fokus. Positionsbestimmungen und Perspektiven. ed. by Ludwig M. Eichinger. 323–352. Berlin/Boston: de Gruyter. DOI logoGoogle Scholar
Deppermann, Arnulf, Streeck, Jürgen. (Eds.). 2018. Time in Embodied Interaction. Amsterdam, Philadelphia: John Benjamins. DOI logoGoogle Scholar
De Stefani, Elwys. 2014. “Rearranging (in) space: On mobility and its relevance for the study of face-to-face interaction”. In Space in Language and Linguistics: Geographical, Interactional, and Cognitive Perspectives, ed. by Peter Auer, Martin Hilpert, Anja Stukenbrock and Benedikt Szmrecsanyi. 434–463. Berlin, Boston: De Gruyter.Google Scholar
. 2021. “If-clauses, their grammatical consequents, and their embodied consequence: making people turn around in guided tours.” Frontiers in Communication 6:661165. DOI logoGoogle Scholar
Duchowski, Andrew T. 2017. Eye Tracking Methodology: Theory and Practice. London: Springer International Publishing. DOI logoGoogle Scholar
Duncan, Starkey. 1972. “Some signals and rules for taking speaking turns in conversations.” Journal of Personality and Social Psychology 23, 283–292. DOI logoGoogle Scholar
Enfield, Nick. 2009. The anatomy of meaning. Speech, gesture, and composite utterances. Cambridge: Cambridge University Press. DOI logoGoogle Scholar
Eriksson, Mats. 2009. “Referring as interaction: On the interplay between linguistic and bodily practices.” Journal of Pragmatics 41(2), 240–262. DOI logoGoogle Scholar
Flom, R., Lee, K., Muir, D. (Eds.). 2007. Gaze-Following. Its Development and Significance. Lawrence Erlbaum, Mahwah, NJ.Google Scholar
Goffman, Erving. 1963. Behaviour in Public Places: Notes on the Social Organization of Gatherings. New York: Free Press.Google Scholar
. 1981. “Footings.” In: Forms of talk. ed. by Erving Goffman. 124–159. Philadelphia: University of Pennsylvania Press.Google Scholar
. 1986. Frame Analysis. An Essay on the Organization of Experience. Boston: Northeastern University Press.Google Scholar
Goodwin, Charles. 1980. “Restarts, Pauses, and the Achievement of a State of Mutual Gaze at Turn-Beginning.” Sociological Inquiry 50 (3–4), 272–302. DOI logoGoogle Scholar
. 1981. Conversational organization: Interaction between speakers and hearers. Academic Press.Google Scholar
. 1984. “Notes on story structure and the organization of participation.” In Structures of social action: Studies in Conversation Analysis, ed. by J. Maxwell Atkinson and John Heritage. 225–246. London: Cambridge University Press.Google Scholar
. 1986. “Between and within: Alternative sequential treatments of continuers and assessments” Human Studies 9(2–3), 205–217. DOI logoGoogle Scholar
Goodwin, Marjorie. 1980. “Processes of Mutual Monitoring Implicated in the Production of Description Sequences.” Sociological Inquiry 50(3–4), 303–317. DOI logoGoogle Scholar
Goodwin, Charles. 2003. “Pointing as situated practice.” In Pointing: Where Language, Culture, and Cognition Meet. ed. by Satoro Kita. 217–241. Lawrence Erlbaum, Mahwah, NJ.Google Scholar
. 1994. “Professional Vision.” American Anthropologist. 96(3), 606–633. DOI logoGoogle Scholar
. 1996. “Transparent Vision.” In: Interaction and Grammar. ed. by Elinor Ochs, Emanuel A. Schegloff und Sandra A. Thompson. 370–404. Cambridge: Cambridge University Press. DOI logoGoogle Scholar
Goodwin, Marjorie and Goodwin, Charles. 1986. “Gesture and coparticipation in the activity of searching for a word.” Semiotica 62, 51–75.Google Scholar
Hindmarsh, Jon and Heath, Christian. 2000. “Embodied reference: A study of deixis in workplace interaction.” Journal of Pragmatics 32(12), 1855–1878. DOI logoGoogle Scholar
Holler, Judith and Kendrick, Kobin. 2015. “Unaddressed participants’ gaze in multi-person interaction: Optimizing recipiency.” Frontiers in Psychology 6: 98. DOI logoGoogle Scholar
Holler, Judith and Levinson, Stephen C. 2019. “Multimodal Language Processing in Human Communication.” Trends in Cognitive Science 23(8), 639–652. DOI logoGoogle Scholar
Holmqvist, Kenneth, Nyström, Marcus, Andersson, Richard, Dewhurst, Richard, Halszka, Jarodzka, and Joost van de Weijer (eds). 2011. Eye Tracking: A Comprehensive Guide to Methods and Measures. Oxford: Oxford University Press.Google Scholar
Jehoul, Annelies, Brône, Geert and Feyaerts, Kurt. 2017. “Gaze patterns and fillers. Empirical data on the difference between Dutch ‘euh’ and ‘euhm’.” Proceedings of the 4th European and 7th Nordic Symposium on Multimodal Communication (MMSYM 2016), 43–50.Google Scholar
Jucker, Andreas H., Schneider, Klaus P., Bublitz, Woldfram. Eds. 2018. Methods in Pragmatics. De Gruyter Mouton, Berlin, Boston. DOI logoGoogle Scholar
Jucker, Andreas H., Taavitsainen, Irma (Eds.). 2010. Historical Pragmatics. Berlin, Bosotn: De Gruyter Mouton, Berlin, Boston. DOI logoGoogle Scholar
Kendon, Adam. 1967. “Some functions of gaze direction in social interaction.” Acta Psychologica 26: 22–63. DOI logoGoogle Scholar
. 1990. “Movement coordination in social interaction: some examples described.“ In Conducting interaction. Patterns of behavior in focused encounters. ed. by Adam Kendon. 91–115. Cambridge: Cambridge University Press.Google Scholar
Kendrick, Kobin H. and Holler, Judith. 2017. “Gaze Direction Signals Response Preference in Conversation.” Research on Language and Social Interaction 50(1), 12–32. DOI logoGoogle Scholar
Kristiansen, Elisabeth Dalby and Rasmussen, Gitte. 2021. “Eye-tracking Recordings as Data in EMCA Studies: Exploring Possibilities and Limitations. Social Interaction.” Video-Based Studies of Human Sociality 4(4). DOI logoGoogle Scholar
Krug, Maximilian. 2020. “Erzählen inszenieren. Ein Theatermonolog als multimodale Leistung des Interaktionsensembles auf der Probebühne.“ Linguistik online 104 (4), 59–81. DOI logoGoogle Scholar
. 2022. Gleichzeitigkeit in der Interaktion. Strukturelle (In)Kompatibilität bei Multiaktivitäten in Theaterproben. Berlin: De Gruyter. DOI logoGoogle Scholar
Locher, Miriam A., Graham, Sage L. Eds. 2010. Interpersonal Pragmatics. Berlin, Boston: Mouton de Gruyter. DOI logoGoogle Scholar
Mondada, Lorenza. 2013. “Conversation analysis: Talk and bodily resources for the organization of social interaction”. In: Body — Language — Communication. Volume 1. ed. by Cornelia Müller, Alan Cienki, Ellen Fricke, Silva Ladewig, David McNeill and Sedinha Tessendorf, 218–226. Berlin, Boston: De Gruyter Mouton. DOI logoGoogle Scholar
. 2014. “Bodies in action: multimodal analysis of walking and talking.” Language and Dialogue 4 (3): 357–403. DOI logoGoogle Scholar
. 2019. “Contemporary Issues in Conversation Analysis: Embodiment and Materiality, Multimodality and Multisensoriality in Social Interaction.” Journal of Pragmatics 145: 47–62. DOI logoGoogle Scholar
Nishizaka, Aug. 2017. “The Perceived Body and Embodied Vision in Interaction”. Mind, Culture, and Activity 24(2), 110–128. DOI logoGoogle Scholar
Oben, Bert. 2018. “Gaze as a predictor for lexical and gestural alignment.” In Eye-Tracking in interaction. Studies on the role of eye gaze in dialogue. ed by Geert Brône and Bert Oben. 233–264. Amsterdam/Philadelphia: John Benjamins. DOI logoGoogle Scholar
Oben, Bert, Brône, Geert. 2016. “Explaining interactive alignment: A multimodal and multifactorial account.“ Journal of Pragmatics, 104: 32–51. DOI logoGoogle Scholar
Pfeiffer, Martin and Weiss, Clarissa. 2022. “Reenactments during tellings: Using gaze for initiating reenactments, switching roles, and representing events.“ Journal of Pragmatics 189, 92–113. DOI logoGoogle Scholar
Płużyczka, Monika. 2018. “The First Hundred Years: a History of Eye Tracking as a Research Method.“ Applied Linguistics Papers. 25/4, 101–116. DOI logoGoogle Scholar
Rossano, Federico. 2012a. Gaze behaviour in face-to-face conversation [Doctoral dissertation]. Radboud University Nijmegen.
. 2012b. “Gaze in Conversation.” The Handbook of Conversation Analysis. ed. by Jack Sidnell and Tanya Stivers: 308–329. Oxford, U.K.: Wiley Blackwell. DOI logoGoogle Scholar
Rühlemann, Christoph. 2022. “How is emotional resonance achieved in storytellings of sadness/distress?” Frontiers in Psychology Volume 13 — 2022. DOI logoGoogle Scholar
Rühlemann, Christoph, and Alexander Ptak. 2023. “Reaching beneath the Tip of the Iceberg: A Guide to the Freiburg Multimodal Interaction Corpus.” Open Linguistics 9 (1). DOI logoGoogle Scholar
Sacks, Harvey. 1992. Lectures on conversation. Vol. 1 & 2 Oxford: Basil Blackwell.Google Scholar
Scaife, Michael and Bruner, Jerome S. 1975. “The capacity for joint visual attention in the infant.” Nature 253(5489), 265–666. DOI logoGoogle Scholar
Schegloff, Emanuel A. 1982. “Discourse as an interactional achievement: Some uses of ‘uh huh’ and other things that come between sentences.” In Analyzing Discourse: Text and Talk, ed. by Deborah Tannen, 71–93. Georgetown University Press.Google Scholar
Schütz, Alfred. 1953. “Common-sense and Scientific Interpretation of Human Action.” Philosophy and Phenomenological Research 14 (1): 1–38. DOI logoGoogle Scholar
Selting, Margret, Auer, Peter, Barth-Weingarten, Dagmar, Bergmann, Jörg, Bergmann, Pia, Birkner, Karin, Couper-Kuhlen, Elizabeth, Deppermann, Arnulf, Gilles, Peter, Günthner, Susanne, Hartung, Martin, Kern, Frederike, Mertzlufft, Christine, Meyer, Christian, Morek, Miriam, Oberzaucher, Frank, Peters, Jörg, Quasthoff, Uta, Schutte, Wilfried, Stukenbrock, Anja, and Uhmann, Susanne. 2009. “Gesprächsanalytisches Transkriptionssystem 2 (GAT 2).“ Gesprächsforschung — Online-Zeitschrift zur verbalen Interaktion 10: 353–402.Google Scholar
Stivers, Tanya. 2010. “An overview of the question–response system in American English conversation.” Journal of Pragmatics 42(10), 2772–2781. DOI logoGoogle Scholar
Stivers, Tanya and Federico Rossano. 2010. “Mobilising response.” Research on Language and Social Interaction 43: 3–31. DOI logoGoogle Scholar
Stivers, Tanya, Enfield, Nick., Brown, Penelope, Englert, Christona, Hayashi, Makoto, Heinemann, Trine, Hoymann, Gertie, Rossano, Frederico, De Ruiter, Jan P., Yoon, Kyong-Eun and Levinson, Steven C. 2009. “Universals and cultural variation in turn-taking in conversation.“ Proceedings of the National Academy of Sciences 106(26), 10587–10592. DOI logoGoogle Scholar
Streeck, Jürgen. 2014. “Mutual gaze and recognition: Revisiting Kendon’s ‘Gaze direction on two-person interaction’“ In From Gesture in Conversation to Gesture as Visible Utterance: Essays in Honor of Adam Kendon. ed. by Mandana Seyfeddinipur and Marianne Gullberg. 35–55. Amsterdam: Benjamins.Google Scholar
Streeck, Jürgen, Goodwin, Charles, LeBaron, Curtis. Eds. (2011). Embodied Interaction: Language and Body in the Material World. Cambridge: Cambridge University Press.Google Scholar
Stukenbrock, Anja. 2009. “Referenz durch Zeigen. Zur Theorie der Deixis.“ Deutsche Sprache 37, 289–316. DOI logoGoogle Scholar
. 2014a. “Take the words out of my mouth: Verbal instructions as embodied practices.” Journal of Pragmatics 65, 80–102. DOI logoGoogle Scholar
. 2014b. “Pointing to an ‘empty’ space: Deixis am Phantasma in face-to-face interaction.” Journal of Pragmatics 74, 70–93. DOI logoGoogle Scholar
. 2015. Deixis in der face-to-face-Interaktion. Berlin, Boston: De Gruyter. DOI logoGoogle Scholar
. 2018a. “Mobile dual eye-tracking in face-to-face interaction. The case of deixis and joint attention.“ In Eye Tracking in Interaction. Studies on the Role of Eye Gaze in Dialogue. ed. by Geert Brône and Bert Oben. 265–301. Amsterdam: Benjamins. DOI logoGoogle Scholar
. 2018b. “Blickpraktiken von Sprechern und Hörern bei der Lokaldeixis: Mobile Eye Tracking-Analysen.“ Gesprächsforschung. Online-Zeitschrift zur verbalen Interaktion, 132–168.Google Scholar
. 2020. “Deixis, Meta-Perceptive Gaze Practices, and the Interactional Achievement of Joint Attention.” Frontiers in Psychology 11, 1779–1779. DOI logoGoogle Scholar
. 2023. “Temporality and the cooperative infrastructure of human communication: Noticings to delay and to accelerate onward movement in mobile interaction“. Language & Communication 92, 33–54. DOI logoGoogle Scholar
Stukenbrock, Anja and Dao, Anh Nhi. 2019. “Joint Attention in Passing: What Dual Mobile Eye Tracking Reveals About Gaze in Coordinating Embodied Activities at a Market: Social Encounters in Time and Space”. In Embodied Activities in Face-to-face and Mediated Settings, ed. by Elisabeth Reber and Cornelia Gerhardt. 177–213. Palgrave Macmillan. DOI logoGoogle Scholar
Sweetser, Eve and Stec, Kashmiri. 2016. “Maintaining multiple viewpoints with gaze.” In: Viewpoint and the Fabric of Meaning: Form and Use of Viewpoint Tools across Languages and Modalities. ed. by Barbara Dancygier, Wie-lun Lu and Arie Verhagen. 237–258. Berlin: de Gruyter Mouton. DOI logoGoogle Scholar
Tomasello Michael, Hare, Brian, Lehmann Hagen, and Call, Josep. 2007. “Reliance on head versus eyes in the gaze following of great apes and human infants: the cooperative eye hypothesis.“ Journal of Human Evolution 52(3), 314–20. DOI logoGoogle Scholar
Vranjes, Jelena, Brône, Geert and Feyaerts, Kurt. 2018. “Dual feedback in interpreter-mediated interactions: On the role of gaze in the production of listener responses.” Journal of Pragmatics 134, 15–30. DOI logoGoogle Scholar
Wade, Nicholas and Tatler, Benjamin. 2009. “Did Javal measure eye movements during reading?” Journal of Eye Movement Research 5, 5–7. DOI logoGoogle Scholar
Weiß, Clarissa. 2018. “When gaze-selected next speakers do not take the turn.” Journal of Pragmatics 133, 28–44. DOI logoGoogle Scholar
. 2019. “Blickverhalten des nicht-blickselegierten Sprechers während Korrekturen und Elaborierungen.“ Gesprächsforschung — Online-Zeitschrift zur verbalen Interaktion 20, 1–28.Google Scholar
Weiß. 2020. Blick und Turn-Taking in Face-to-Face-Interaktionen: Multimodale Interaktionsanalysen triadischer Gesprächssituationen mit Hilfe von Eye-Tracking. Verlag für Gesprächsforschung. [URL]
Wittenburg, P., Brugman, H., Russel, A., Klassmann, A. & H. Sloetjes. 2006. “ELAN: a Professional Framework for Multimodality Research.” Proceedings of LREC 2006, Fifth International Conference on Language Resources and Evaluation, 1556–1559.Google Scholar
Zima, Elisabeth. 2018. “Multimodale Mittel der Rederechtsaushandlung im gemeinsamen Erzählen.“ [Multimodal means to negotiate turn-taking in joint storytellings] Gesprächsforschung — Online — Zeitschrift zur verbalen Interaktion (18), 241–273.Google Scholar
. 2020. “Gaze and recipient feedback in triadic storytelling activities.” Discourse Processes 57 (9), 725–748. DOI logoGoogle Scholar
Zima, Elisabeth, Weiß, Clarissa and Geert Brône. 2019. “Gaze and overlap resolution in triadic interactions.” Journal of Pragmatics 140, 49–69. DOI logoGoogle Scholar