Cover not available
Part of
Mobile Eyetracking: New avenues for the study of gaze in social interaction
Edited by Elisabeth Zima and Anja Stukenbrock
[Pragmatics & Beyond New Series 351] 2025
► pp. 101–128

Chapter 4
Mobile eye-tracking and mixed-methods approaches to interaction analysis

 | University of Leuven
 | University of Leuven
 | University of Leuven
The integration of mobile eye-tracking technology in linguistic research has catalyzed a surge of investigations across diverse linguistic subdisciplines. This chapter advocates for a mixed-methods approach in analyzing eye gaze behavior during face-to-face interactions. Through two case studies, examining eye gaze in interactional irony and gaze synchronization, we demonstrate how this approach can help enhance our understanding of conversational eye gaze behaviour.
Keywords: eye gaze, interaction, mixed-methods, irony, synchronization
Article outline
  • 1.Introduction
  • 2.Defining and refining units of analysis: A case study on irony in interaction
  • 3.Mutual gaze during face-to-face interaction: A second case study
    • 3.1Data and method
    • 3.2Results
    • 3.3Discussion of the quantitative results
    • 3.4Further explaining the observed synchronisation in qualitative observations
    • 3.5Functional quantification
  • 4.Conclusion
  • Author queries
  • Notes
  • References
This content is being prepared for publication; it may be subject to changes.
References (59)
References
Argyle, Michael and Cook, Mark. 1976. Gaze and Eye contact. London: Cambridge University Press.Google Scholar
Auer, Peter. 2005. “Projection in Interaction and Projection in Grammar.” Text — Interdisciplinary Journal for the Study of Discourse 25(1), 7–36. DOI logoGoogle Scholar
. 2021. “Turn-allocation and gaze: A multimodal revision of the “current-speaker-selects-next” rule of the turn-taking system of conversation analysis.” Discourse Studies 23(2), 117–140. DOI logoGoogle Scholar
Auer, Peter & Zima, Elisabeth. 2021. “On word searches, gaze, and co-participation.” Gesprächsforschung — Online — Zeitschrift zur verbalen Interaktion 22, 390–425.Google Scholar
Barnden, John A. 2017. “Irony, pretence and fictively-elaborating hyperbole.“ In Irony in language use and communication, ed. by Anegliki Athanasiadou and Herbert L. Colston, 145–178. Amsterdam: John Benjamins. DOI logoGoogle Scholar
Bavelas, Janet and Chovil, Nicole. 2018. Some pragmatic functions of conversational facial gestures. Gesture 17, 98–127. DOI logoGoogle Scholar
Bavelas, Janet, Coates, Linda and Johnson, Trudy. 2002. Listener responses as a collaborative process: The role of gaze. Journal of Communication 52, 566–580. DOI logoGoogle Scholar
Bishop, Laura, Cancino-Chacon, Carlos, and Goebl, Werer. 2021. “Beyond synchronization: Body gestures and gaze direction in duo performance.” In Together in Music: Participation, Co-Ordination, and Creativity in Ensembles. ed. by Renee Timmers, Freya Bailes, and Helena Daffern, 182–188. Oxford: Oxford University. DOI logoGoogle Scholar
Brennan, Susan, Chen, Xin, Dickinson, Christopher A., Neider, Mark B., and Zelinsky, Gregory. 2008. “Coordinating cognition: The costs and benefits of shared gaze during collaborative search.” Cognition 106, 1465–1477. DOI logoGoogle Scholar
Brône, Geert. 2008. “Hyper- and misunderstanding in interactional humour”. Journal of Pragmatics 40(12), 2027–2061. DOI logoGoogle Scholar
. 2021. “The multimodal negotiation of irony and humour in interaction: On the role of eye gaze in joint pretense”. In Figurative Thought and Language (Vol. 11), ed. by Augusto Soares da Silva, 109–136. Amsterdam: John Benjamins Publishing Company.Google Scholar
Brône, Geert and Oben, Bert. 2015. “InSight Interaction: A multimodal and multifocal dialogue corpus.” Language Resources and Evaluation 49(1), 195–214. DOI logoGoogle Scholar
Brône, Geeert and Oben, Bert (Eds.) 2018. Eye-tracking in Interaction: Studies on the role of eye gaze in dialogue. (Advances in Interaction Studies (AIS)). Amsterdam: John Benjamins. DOI logoGoogle Scholar
Brône, Geert and Oben, Bert. 2022. “Monitoring the pretence. Intersubjective grounding, gaze and irony.“ In Cognitive Sociolinguistics Revisited. ed. by Gitte Kristiansen, Karlien Franco, Stefano De Pascale, Laura Rosseel and Weiwei Zhang, 544–556. Berlin: De Gruyter Mouton.Google Scholar
Brône, Geert & Oben, Bert. 2023. Mobile eye-tracking for multimodal interaction analysis. In The Routledge Handbook of Experimental Linguistics. ed. by Sandrine Zufferey and Pascal Gygax, 283–298. London: Routledge. DOI logoGoogle Scholar
Chartrand, Tanya L. and Bargh, John A. 1999. “The chameleon effect: The perception behaviour link and social interaction.” Journal of Personality and Social Psychology 76(6), 893–910. DOI logoGoogle Scholar
Clark, Herbert H. 1996. Using Language. Cambridge University Press. DOI logoGoogle Scholar
2016. “Depiction as a method of communication.” Psychological Review 123–3, 324–347. DOI logoGoogle Scholar
Clark, Herbert H. & Gerrig, Richard. 1984. On the pretense theory of irony. Journal of Experimental Psychology: General 113, 121–126. DOI logoGoogle Scholar
Coco, Moreno I. and Dale, Rick. 2014. “Cross-recurrence quantification analysis of categorical and continuous time series: an R package.” Frontiers in Psychology 5, 510. DOI logoGoogle Scholar
Coulson, Seana. 2005. “Sarcasm and the space-structuring model.” In The Literal and Nonliteral in Language and Thought. ed. by Seana Coulson and Barbara Lewandowska-Tomaszczyk. 129–144. Frankfurt: Peter Lang.Google Scholar
Cummins, Fred. 2011. “Gaze and blinking in dyadic conversation: A study in coordinated behaviour among individuals.” Language and Cognitive Processes 27, 1525–1549. DOI logoGoogle Scholar
Dale, Rick, Kirkham, Natasha and Richardson, Daniel. 2011. “How two people become a tangram recognition system.” Proceedings of the European Conference on Computer-Supported Cooperative Work. Berlin: Springer Verlag.Google Scholar
de Ruiter, Jan, Mitterer, Holger and Enfield, Nick. 2006. “Predicting the end of a speaker’s turn: A cognitive cornerstone of conversation.” Language 82, 515–535. DOI logoGoogle Scholar
De Stefani, Elwys. 2021. “Embodied responses to questions-in-progress: silent nods as affirmative answers.” Discourse Processes 58(4), 353–371. DOI logoGoogle Scholar
de Vries, Clarissa, Oben, Bert and Brône, Geert. 2021. “Exploring the role of the body in communicating ironic stance.” Languages and Modalities 1(1), 65–80. DOI logoGoogle Scholar
ELAN (Version 6.4) [Computer software]. 2023. Nijmegen: Max Planck Institute for Psycholinguistics, The Language Archive. Retrieved from [URL]
Frischen, Alexandra, Bayliss, Andrew P. and Tipper, Steven. 2007. “Gaze cueing of attention: Visual attention, social cognition, and individual differences.” Psychological Bulletin 133(4), 694–724. DOI logoGoogle Scholar
Fusaroli, Riccardo and Tylén, Kristian. 2016. “Investigating Conversational Dynamics: Interactive Alignment, Interpersonal Synergy, and Collective Task Performance.” Cognitive Science 40(1), 145–71. DOI logoGoogle Scholar
Fusaroli, Riccardo, Konvalinka, Ivana and Wallot, Sebastian. 2014. “Analyzing social interactions: The promises and challenges of using cross recurrence quantification analysis.” Springer Proceedings in Mathematics and Statistics 103, 137–155. DOI logoGoogle Scholar
Gironzetti, Elisa, Pickering, Lucy, Huang, Meichan, Zhang, Ying, Menjo, Shigehito and Attardo, Salvatore. 2016. “Smiling synchronicity and gaze patterns in dyadic humourous conversations.” Humour: International Journal of Humour Research 29(2), 301–324.Google Scholar
González-Fuente, Santoago, Escandell-Vidal, Victoria and Prieto, Pilar. 2015. “Gestural codas pave the way to the understanding of verbal irony.” Journal of Pragmatics 90, 26–47. DOI logoGoogle Scholar
Goodwin, Marjorie and Goodwin, Charles. 1986. “Gesture and coparticipation in the activity of searching for a word.” Semiotica 62, 51–75.Google Scholar
Hadelich, Kerstin and Crocker, Matthew. 2006. “Gaze alignment of interlocutors in conversational dialogues.” Proceedings of the 2006 Symposium on Eye Tracking Research and Applications, 38. DOI logoGoogle Scholar
Heller, Vivien. 2021. Embodied displays of “Doing Thinking.” Epistemic and interactive functions of thinking displays in children’s argumentative activities. Frontiers in Psychology 12: 636671. DOI logoGoogle Scholar
Jehoul, Annelies, Brône, Geert and Feyaerts, Kurt. 2017. “Gaze patterns and fillers. Empirical data on the difference between Dutch ‘euh’ and ‘euhm’.” Proceedings of the 4th European and 7th Nordic Symposium on Multimodal Communication (MMSYM 2016), 43–50.Google Scholar
Kendon, Adam. 1967. “Some functions of gaze-direction in social interaction.” Acta Psychologica 26, 22–63. DOI logoGoogle Scholar
Kendrick, Kobin H. and Holler, Judith. 2017. “Gaze Direction Signals Response Preference in Conversation.” Research on Language and Social Interaction 50(1), 12–32. DOI logoGoogle Scholar
Lachat, Fanny, Conty, Laurence, Hugueville, Laurent and George, Nathalie. 2012. “Gaze cueing effect in a face-to-face situation.” Journal of Nonverbal Behaviour 36, 177–190. DOI logoGoogle Scholar
Louwerse, Max, Dale, Rick, Bard, Ellen and Jeuniaux, Patrick. 2012. “Behaviour matching in multimodal communication is synchronized.” Cognitive Science 36, 1404–1426. DOI logoGoogle Scholar
Mondada, Lorenza. 2014a. “The local constitution of multimodal resources for social interaction.” Journal of Pragmatics 65, 137–156. DOI logoGoogle Scholar
. 2014b. “Pointing, talk and the bodies: reference and joint attention as embodied interactional achievements” In From Gesture in Conversation to Visible Utterance in Action, ed. by Mandana Seyfeddinipur and Marianne Gullberg, 95–124. Amsterdam: John Benjamins, 95–124. DOI logoGoogle Scholar
Nakano, Tamami and Kitazawa, Shigeru. 2010. “Eyeblink entrainment at breakpoints of speech.” Experimental Brain Research 205, 577–581. DOI logoGoogle Scholar
Neider, Mark B., Chen, Xin, Dickinson, Christopher, Brennan, Susan and Zelinsky, Gregory. 2010. “Coordinating spatial referencing using shared gaze.” Psychonomic Bulletin & Review 17, 718–724. DOI logoGoogle Scholar
Oertel, Chatarine, Wlodarczak Marcin, Edlund Jens, Wagner Petra and Gustafson Joakim. 2012. “Gaze patterns in turn-taking.” Proceedings of Interspeech, 2247–2250. DOI logoGoogle Scholar
Richardson, Daniel and Dale, Rick. 2005. “Looking to understand: The coupling between speakers’ and listeners’ eye movements and its relationship to discourse comprehension.” Cognitive Science 29, 1045–1060. DOI logoGoogle Scholar
Richardson, Daniel, Dale, Rick and Tomlinson, John. 2009. “Conversation, gaze coordination, and beliefs about visual context.” Cognitive Science 33, 1468–1482. DOI logoGoogle Scholar
Sacks, Harvey, Schegloff, Emanuel. and Jefferson, Gail. 1974. “A simplest systematics for the organization of turn-taking for conversation.” Language 50, 696–735. DOI logoGoogle Scholar
Shockley, Kevin, Santana, Marie-Vee and Fowler, Carol. 2003. “Mutual interpersonal postural constraints are involved in cooperative conversation.” Journal of Experimental Psychology: Human Perception and Performance 29, 326–332.Google Scholar
Sperber, Dan and Wilson, Deirdre. 1981. “Irony and the use-mention distinction.” In Radical Pragmatics. ed. by Peter Cole. 295–318. New York: Academic Press.Google Scholar
Stukenbrock, Anja. 2020. “Deixis, Meta-Perceptive Gaze Practices, and the Interactional Achievement of Joint Attention.” Frontiers in Psychology 11: 1779. DOI logoGoogle Scholar
Tabacaru, Sabina and Lemmens, Maarten. 2014. “Raised eyebrows as gestural triggers in humour: The case of sarcasm and hyper-understanding.” The European Journal of Humour Research 2(2), 11–31. DOI logoGoogle Scholar
Ten Bosch, Louis, Oostdijk, Nelleke and Boves, Lou. 2005. “On temporal aspects of turn taking in conversational dialogues.” Speech Communication 47(1), 80–86. DOI logoGoogle Scholar
Tobin, Vera. 2016. Performance, Irony, and Viewpoint in Language. In Theatre, Performance and Cognition: Languages, Bodies and Ecologies. ed. by Amy Cook and Ronda Blair. 54–67. London: Bloomsbury Methuen. DOI logoGoogle Scholar
Vandemoortele, Sarah, Feyaerts, Kurt, Reybrouck, Mark, De Bièvre, Geert, Brône, Geert, De Baets, Thomas. 2018. “Gazing at the partner in musical trios: a mobile eye-tracking study.” Journal of Eye Movement Research 11 (2), 1–13. DOI logoGoogle Scholar
Vrzakova, Hana, Amon, Mary J., Stewart, Angela and D’Mello, Sidney. 2019. “Dynamics of Visual Attention in Multiparty Collaborative Problem Solving using Multidimensional Recurrence Quantification Analysis.” Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (CHI ‘19), Paper 342, 1–14. DOI logoGoogle Scholar
Wallot, Sebastian. 2017. “Recurrence quantification analysis of processes and products of discourse: A tutorial in R.” Discourse Processes 54(5–6), 382–405. DOI logoGoogle Scholar
Xu, Tian, de Barbaro, Kaya, Abney, Drew and Cox, Ralf. 2020. “Finding Structure in Time: Visualizing and Analyzing Behavioural Time Series.” Frontiers in Psychology 11, 1457. DOI logoGoogle Scholar
Zima, Elisabeth, Weiß, Clarissa and Geert Brône. 2019. “Gaze and overlap resolution in triadic interactions.” Journal of Pragmatics 140, 49–69. DOI logoGoogle Scholar