Cover not available
Part of
Mobile Eyetracking: New avenues for the study of gaze in social interaction
Edited by Elisabeth Zima and Anja Stukenbrock
[Pragmatics & Beyond New Series 351] 2025
► pp. 68–100

Chapter 3
The influence of the specificities of gaze behavior on emerging and ensuing interaction

A contribution to the discussion of the use of eye-tracking recordings for EMCA analysis

 | University of Southern Denmark
 | University of Southern Denmark
The integration of new technologies in Ethnomethodology and Conversation Analysis necessitates thorough discussion. This chapter explores the combination of recordings from a mobile eye-tracking device with recordings from an external mobile video camera, which may reveal intricate details of human activities. Focusing on customers’ actions and interactions with salespersons, the chapter demonstrates how even brief observations made by customers, as captured by the eye trackers, are significant in understanding their subsequent actions when navigating amongst one another. In addition, it illustrates how customers’ initiations and responses to salespersons’ initiations of talk are to be understood in the context of the specificities of their prior observations. The main point emphasized is that eye-tracking recordings, along with video recordings from external cameras, capture essential behavioral nuances, leading to re‑specifications of aspects of social action and interaction.
Keywords: eye-tracking data, video recordings, search activities, observations, conversation initiation, embodied actions
Article outline
  • 1.Introduction
    • 1.1Research on pre-activities and pre-sequences
  • 2.Data collection
  • 3.Customers’ perceptions and their relation to subsequent embodied conduct
  • 4.Customers’ perceptions and their relation to sequence initiations and responses
    • 4.1Search activities and their relation to recruitment sequences
  • 5.Discussion
  • Author queries
  • Notes
  • References
  • Appendix
This content is being prepared for publication; it may be subject to changes.
References (56)
References
Auer, Peter. 2018. “Gaze, addressee selection and turn-taking in three-party interaction.” In Eye-tracking in Interaction: Studies on the role of eye gaze in dialogue, ed. by Geert Brône, and Bert Oben, 197–232. Amsterdam/Philadelphia: John Benjamins Publishing Company. DOI logoGoogle Scholar
. 2021. “Turn-allocation and gaze: A multimodal revision of the “current-speaker-selects-next” rule of the turn-taking system of conversation analysis.” Discourse Studies 23 (2): 117–140. DOI logoGoogle Scholar
Brône, Geert and, Bert Oben (eds). 2018. Eye-tracking in Interaction: Studies on the role of eye gaze in dialogue. John Benjamins Publishing Company. DOI logoGoogle Scholar
Broth, Mathias, and Lorenza Mondada. 2013. “Walking away: The embodied achievement of activity closings in mobile interaction.” Journal of Pragmatics 47 (1): 41–58. DOI logoGoogle Scholar
Carter, Benjamin T., and Steven G. Luke. 2020. “Best practices in eye tracking research.” International Journal of Psychophysiology 155: 49–62. DOI logoGoogle Scholar
Ciolek, Matthew T., and Adam Kendon. 1980. “Environment and the Spatial Arrangement of Conversational Encounters.” Sociological Inquiry 50 (3): 237–271. DOI logoGoogle Scholar
Day, Dennis, and Gitte Rasmussen. 2019. “Interactional Consequences of Object Possession in Institutional Practices.” In Objects, Bodies and Work Practice, ed. by Dennis Day, and Johs. Wagner, 87–112. Blue Ridge Summit, PA: Multilingual Matters.Google Scholar
Day, Dennis, and Johs. Wagner (eds.). 2019. Objects, Bodies, and Work Practice. Blue Ridge Summit, PA: Multilingual Matters.Google Scholar
Deppermann, Arnulf. 2013a. “Turn-design at turn-beginnings: Multimodal resources to deal with tasks of turn-construction in German.” Journal of Pragmatics 46: 91–121. DOI logoGoogle Scholar
. 2013b. “Conversation Analytic Studies of Multimodal Interaction.” Journal of Pragmatics 46 (1): 1–172. DOI logoGoogle Scholar
De Stefani, Elwys. 2014. “Establishing joint orientation towards commercial objects in a self-service store.” In Interacting with Objects: Language, materiality, and social activity, ed. by Maurice Nevile, Pentti Haddington, Trine Heinemann, and Mirka Rauniomaa, 271–294. Amsterdam/Philadelphia: John Benjamins Publishing Company. DOI logoGoogle Scholar
De Stefani, Elwys, and Lorenza Mondada. 2010. „Die Eröffnung sozialer Begegnungen im öffentlichen Raum: Die emergente Koordination räumlicher, visueller und verbaler Handlungsweisen.“ In Situationseröffnungen Zur multimodalen Herstellung fokussierter Interaktion, ed. by Lorenza Mondada, and Reinhold, Schmitt R., 103–170. Tübingen: narr verlag.Google Scholar
. 2014. “Reorganizing mobile formations: When “guided” participants initiate reorientations in guided tours.” Space and Culture 17 (2): 157–175. DOI logoGoogle Scholar
. 2018. “Encounters in Public Space: How Acquainted Versus Unacquainted Persons Establish Social and Spatial Arrangements.” Research on Language and Social Interaction 51 (3): 248–270. DOI logoGoogle Scholar
Garfinkel, Harold. 1967. Studies in Ethnomethodology. Englewood Cliffs, NJ: Prentice Hall.Google Scholar
Garfinkel, Harold, and Harvey Sacks. 1986. “On formal structures of practical actions.” In Ethnomethodological studies of work ed. by Harold Garfinkel, 160–193. Routledge & Kegan Paul.Google Scholar
Garfinkel, Harold, and D. Lawrence Wieder. 1992. “Two incommensurable asymmetrically alternate technologies of social analysis”. In Text in Context: Contributions to Ethnomethodology, ed. by Graham Watson, and Robert Morris Seiler, 175–206. Sage Publications.Google Scholar
Goffman, Erving. 1981. Forms of Talk. Philadelphia: University of Pennsylvania Press.Google Scholar
Goodwin, Charles. 1980. “Restarts, pauses, and the achievement of a state of mutual gaze at turn-beginning.” Sociological Inquiry 50 (3–4): 272–302. DOI logoGoogle Scholar
. 1981. Conversational organization: Interaction between speakers and hearers. New York: Academic Press.Google Scholar
. 1994. “Professional Vision.” American Anthropologist 96 (3): 606–633. DOI logoGoogle Scholar
Heath Christian, Jon Hindmarsh, and Paul Luff. 2010. Video in Qualitative Research: Analysing Social Interaction in Everyday Life. London: Sage.Google Scholar
Kendon, Adam. 1973. “The role of visible behaviour in the organisation of social interaction.” In Social communication and movement, ed. by Marion Carnach, and Ian Vine, 29–74. London: Academic Press.Google Scholar
. 1990. Conducting interaction: Patterns of behaviour in focused encounters. CUP Archive.Google Scholar
Kendon, Adam, and Andrew Ferber. 1973. “A description of some human greetings.” In Comparative ecology and behaviour of primates, ed. by Richard P. Michael, and John Crook, 591–668. London/New York.Google Scholar
Kendrick, Kobin H., and Paul Drew. 2016. “Recruitment: Offers, requests, and the organization of assistance in interaction.” Research on Language and Social Interaction 49: 1–19. DOI logoGoogle Scholar
Knoblauch, Hubert, Bernt Schnettler, Jürgen Raab, and Hans-Georg Soeffner (eds.). 2006. Video Analysis: Methodology and Methods. Qualitative Audiovisual Data Analysis in Sociology. Peter Lang. DOI logoGoogle Scholar
Kristiansen, Elisabeth Dalby, and Gitte Rasmussen. 2021. “Eye-tracking Recordings as Data in EMCA Studies: Exploring Possibilities and Limitations.” Social Interaction. Video-Based Studies of Human Sociality 4 (4). DOI logoGoogle Scholar
LaBonte, Andrew, Jon Hindmarsh, and Dirk vom Lehn. 2021. “Data collection at height: Embodied competence, multisensoriality, and video-based research in an extreme context of work.” Social Interaction. Video-Based Studies of Human Sociality 4 (3). DOI logoGoogle Scholar
Lindström, Jan. 2006. “Grammar in the service of interaction: Exploring turn organization in Swedish.” Research on Language and Social Interaction 39 (1): 81–117. DOI logoGoogle Scholar
McIlvenny, Paul. 2019. “Inhabiting Spatial Video and Audio Data: Towards a Scenographic Turn in the Analysis of Social Interaction.” Social Interaction. Video-based Studies of Human Sociality, 2 (1). DOI logoGoogle Scholar
McIllvenny, Paul, Mathias Broth, and Pentti Haddington. 2014. “Moving Together: Mobile Formations in Action.” Space and Culture 17 (2): 104–106. DOI logoGoogle Scholar
Mondada, Lorenza. 2007. “Multimodal resources for turn-taking: Pointing and the emergence of possible next speakers.” Discourse studies 9 (2), 194–225. DOI logoGoogle Scholar
. 2009. “Emergent focused interactions in public places: A systematic analysis of the multimodal achievement of a common interactional space.” Journal of Pragmatics 41 (10): 1977–1997. DOI logoGoogle Scholar
. 2011. “Understanding as an embodied, situated, and sequential achievement in interaction.” Journal of Pragmatics 43 (2): 542–552. DOI logoGoogle Scholar
. 2013. “Interactional space and the study of embodied talk-in-interaction.” In Space in language and linguistics: Geographical, interactional, and cognitive perspectives, ed. by Peter Auer, Martin Hilpert, Anja Stukenbrock, and Benedikt Szmrecsanyi, 247–275. Berlin: De Gruyter. DOI logoGoogle Scholar
. 2019a. “Practices for Showing, Looking, and Videorecording: The Interactional Establishment of a Common Focus of Attention.” In Embodied Activities in Face-to-face and Mediated Settings: Social Encounters in Time and Space, ed. by Elisabeth Reber, and Cornelia Gerhardt, 63–104. Springer International Publishing. DOI logoGoogle Scholar
. 2019b. “Rethinking bodies and objects in social interaction: A multimodal and multisensorial approach to tasting.” In Discussing new materialism: Methodological implications for the study of materialities, ed. by Ulrike Tikvah Kissmann, Joost van Loon, 109–134. Wiesbaden: Springer. DOI logoGoogle Scholar
Mondada, Lorenza, and Reinhold Schmitt (eds.). 2010. Situationseröffnungen. Zur multimodalen Herstellung fokussierter Interaktion. Tübingen: Narr Francke Attempto Verlag.Google Scholar
Müller, Cornelia, and Ulrike Bohle. 2007. “Das Fundament fokussierter Interaktion. Zur Vorbereitung und Herstellung von Interaktionsräumen durch körperliche Koordination.” In Koordination. Analysen zur multimodalen Interaktion, ed. by Reinhold Schmitt, 129–165. Tübingen: Narr Francke Attempto Verlag.Google Scholar
Nevile, Maurice, Pentti Haddington, Trine Heinemann, and Mirka Rauniomaa. 2014. Interacting with objects: Language, materiality, and social activity. Amsterdam and Philadelphia: John Benjamins Publishing Company. DOI logoGoogle Scholar
Oloff, Florence. 2010. “Ankommen und Hinzukommen: Zur Struktur der Ankunft von Gästen.“ In Situationseröffnungen Zur multimodalen Herstellung fokussierter Interaktion, ed. by Lorenza Mondada, and Reinhold Schmitt, 171–228. Tübingen: Narr Francke Attempto Verlag.Google Scholar
Rasmussen, Gitte. 2023a. “Analysing interaction involving wheelchairs: configuring interactional spaces to engage in activity and to initiate conversation.” Journal of Interactional Research in Communication Disorders, 14 (2): 328–355. DOI logoGoogle Scholar
. 2023b. “Multimodal engagement and interaction and the appearance of contemporary non-traditional retail shopping.” In Multimodality and Social Interaction in Online and Offline Shopping, ed. by Gitte Rasmussen, and Theo van Leeuwen, 104–127. New York: Routledge. DOI logoGoogle Scholar
Rasmussen, Gitte, and Elisabeth Dalby Kristiansen. 2022. “The sociality of minimizing involvement in Danish self-service shops: Danish customers’ multi-modal practices of being, getting and staying out of the way.” Discourse & Communication 16: 200–232. DOI logoGoogle Scholar
Rasmussen, Gitte, Elisabeth Dalby Kristiansen, and Søren Vigild Poulsen. 2024. E-pub ahead of print. “The World of Daily Life: Doing a search for (e-)shopping purposes.” Pragmatics & Society. DOI logoGoogle Scholar
Rossano, Frederico. 2012. Gaze behaviour in face-to-face interaction, PhD thesis, Radboud University, Nijmegen: The Netherlands.
Rossano, Frederico, Paul Brown, and Stephen Levinson. 2009. “Gaze, questioning and culture.” In Conversation analysis: Comparative perspectives, ed. by Jack Sidnell, 187–249. Cambridge: Cambridge University Press. DOI logoGoogle Scholar
Sacks, Harvey. 1984. “Notes on methodology.” In Structures of social action: Studies in conversation analysis, ed. by Jim M. Atkinson, and John Heritage, 21–27. Cambridge, Paris: Cambridge University Press.Google Scholar
Schegloff, Emanuel A. 1987. “Recycled turn beginnings.” In Talk and Social Organization, ed. by Graham Button, and John R. E. Lee, 70–85. Clevedon: Multilingual Matters. DOI logoGoogle Scholar
1996a. “Confirming Allusions: Toward an Empirical Account of Action.” American Journal of Sociology 102 (1): 161–216. DOI logoGoogle Scholar
Schegloff, Emanuel A., and Gene Lerner. 2009. “Beginning to respond: Well-Prefaced Responses to Wh-Questions.” Journal of Research in Language and Social Interaction 42 (2): 91–115. DOI logoGoogle Scholar
Schmitt, Reinhold, and Arnulf Deppermann. 2007. “Monitoring und Koordination als Voraussetzung der multimodalen Konstitution von Interaktionsräumen.“ In Koordination. Analysen zur multimodalen Interaktion, ed. by Reinhold Schmitt, 95–128. Tübingen: Narr Francke Attempto Verlag.Google Scholar
Stukenbrock, Anja. 2018. “Mobile dual eye-tracking in face-to-face interaction: The case of deixis and joint attention.” In Eye-tracking in Interaction: Studies on the role of eye gaze in dialogue, ed. by Geert Brône, and Bert Oben, 265–300. Amsterdam/Philadelphia: John Benjamins Publishing Company. DOI logoGoogle Scholar
Stukenbrock, Anja, and Anh Nhi Dao. 2019. “Joint attention in passing: What dual mobile eye-tracking reveals about gaze in coordinating embodied activities at a market.” In Embodied Activities in Face-to-face and Mediated Settings, ed. by Elisabeth Reber, and Cornelia Gerhardt, 177–213. Cham: Palgrave MacMillan. DOI logoGoogle Scholar
Wade, Nicholas, and Benjamin W. Tatler. 2005. The Moving Tablet of the Eye: The Origins of Modern Eye Movement Research. Oxford University Press, USA. DOI logoGoogle Scholar