Cover not available
Part of
Eye-tracking in Interaction: Studies on the role of eye gaze in dialogue
Edited by Geert Brône and Bert Oben
[Advances in Interaction Studies 10] 2018
► pp. 1–18

Chapter 1
Introduction

Gaze, interaction and eye-tracking: A multidisciplinary endeavor

Article outline
  • 1.Eye gaze in interaction
  • 2.Eye-tracking in interaction
  • 3.Outline of the volume
    • Part 1: Theoretical considerations
    • Part 2: Methodological considerations
    • Part 3: Case studies
  • Note
  • References
Published online: 13 November 2018
DOI logo
https://doi.org/10.1075/ais.10.01bro
References
Argyle, M. & Cook, M.
(1976) Gaze and mutual gaze. London: Cambridge University Press.Google Scholar
Auer, P.
(2017) Walking and talking. How speakers jointly manoeuvre in space. Invited talk at the international conference on “Spatial Boundaries and Transitions in Language and Interaction”. Monte Veritá, April 2017.
Bailly, G., Raidt, S. & Elisei, F.
(2010) Gaze, conversational agents and face-to-face communication. Speech Communication – special issue on Speech and Face-to-Face Communication 52, 598–612.DOI logoGoogle Scholar
Barisic, I., Timmermans, B., Pfeiffer, U., Bente, G., Vogeley, K., & Schilbach, L.
(2013) In it together: using dual eyetracking to investigate real-time social interactions. Presented at the Proceedings from SIGCHI Conference on Human Factors in Computing Systems, Paris.
Bavelas, J. B., L. Coates & T. Johnson
(2002) Listener responses as a collaborative process: The role of gaze. Journal of Communication, 52, 566–580.DOI logoGoogle Scholar
Brennan, S. E.
(2005) How conversation is shaped by visual and spoken evidence. In J. C. Trueswell & M. Tanenhaus (Eds.), Approaches to studying world-situated language use: Bridging the language-as-product and language-action traditions (pp.95–129). Cambridge, MA: MIT Press.Google Scholar
Brennan S. E., Chen X., Dickinson C., Neider M. & Zelinsky G.
(2008) Coordinating cognition: The costs and benefits of shared gaze during collaborative search. Cognition, 106, 1465–1477.DOI logoGoogle Scholar
Brennan, S. E., Hanna J., Zelinsky G. & Savietta, K.
(2012) Eye gaze cues for coordination in collaborative tasks. Proceedings of the DUET Workshop. Seattle, Feb. 2012.Google Scholar
Brône, G. & Oben, B.
(2015) InSight Interaction. A multimodal and multifocal dialogue corpus. Language Resources and Evaluation, 49, 195–214.DOI logoGoogle Scholar
Brône, G., Oben, B. & Feyaerts, K.
(2010) Introducing the InSight Interaction Corpus. Proceedings of the 7th International Conference on Language Resources and Evaluation. Workshop on Multimodal Corpora. Malta, 17–18 May 2010.Google Scholar
Brône, G., Oben, B., Vranjes, J., Feyaerts, K.
(2017) Eye gaze and viewpoint in multimodal interaction management. Cognitive Linguistics, 28, 449–483.DOI logoGoogle Scholar
Broz, F. et al.
(2012) Mutual gaze, personality, and familiarity: Dual eye-tracking during conversation. Proceedings of HRI 2012.Google Scholar
Clark, A. T., & Gergle, D.
(2011) Mobile Dual Eye-tracking Methods: Challenges and Opportunities. Paper presented at DUET 2011: Dual Eye Tracking workshop at ECSCW 2011.Google Scholar
Clark, H. H.
(1996) Using language, Cambridge: Cambridge University Press.DOI logoGoogle Scholar
De Beugher, S., Brône, G., Goedemé, T.
(2016) Semi-automatic Hand Annotation Making Human-human Interaction Analysis Fast and Accurate. In J. Braz (Ed.), Proceedings of the 11th Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications: Vol. 4. VISIGRAPP. Rome,Italy, 27–29 February 2016 (pp.552–559). Setúbal: Scitepress.DOI logoGoogle Scholar
(2015) Semi-automatic annotation of eye-tracking recordings in terms of human torso, face and hands. In T. Pfeiffer & K. Essig (Eds.), SAGA 2015 Proceedings. International Workshop on Solutions for Automatic Gaze Data Analysis. Bielefeld, Germany, 29–30 September 2015. Bielefeld, Germany: CITEC.Google Scholar
De Jaegher, H., Di Paolo, E. & Gallagher, S.
(2010) Can social interaction constitute social cognition? Trends in Cognitive Sciences, 14(10), 441–447.DOI logoGoogle Scholar
De Ruiter, J. P.
(2005) The role of eye-gaze in visual dialogue tasks. Paper presented at the AMLaP, Ghent, Belgium.Google Scholar
Duncan, S.
(1975) Interaction units during speaking turns in dyadic, face-to-face conversations. In A. Kendon, R. M. Harris & M. R. Key (Eds.), Organization of behavior in face-to-face Interaction (pp.199–212). The Hague: Mouton.DOI logoGoogle Scholar
Duchowski, A.
(2007) Eye tracking methodology. Theory and practice. London: Springer.Google Scholar
Edlund, J., Beskow, J., Elenius, K., Hellmer, K., Strömbergsson, S., House, D.
(2010) Spontal: a Swedish spontaneous dialogue corpus of audio, video and motion capture. In: Proceedings of the Seventh International Conference on Language Resources and Evaluation (LREC)
Emery, N. J.
(2000) The eyes have it: The neuroethology, function and evolution of social gaze. Neuroscience and Biobehavioral Reviews, 24, 581–604.DOI logoGoogle Scholar
Feyaerts, K., Brône, G. & Oben, B.
(2017) Multimodality in Interaction. In: B. Dancygier (Ed.), The Cambridge handbook of cognitive linguistics (pp.135–156). Cambridge: Cambridge University Press.DOI logoGoogle Scholar
Frischen, A., Bayliss, A. P., & Tipper, S. P.
(2007) Gaze cueing of attention: Visual attention, social cognition, and individual differences. Psychological Bulletin, 133, 694–724.DOI logoGoogle Scholar
Goffman, E.
(1981) Forms of talk. Philadelphia: University of Pennsylvania Press.Google Scholar
Goodwin, C.
(1980) Restarts, pauses, and the achievement of a state of mutual gaze. Sociological Inquiry, 272–302.DOI logoGoogle Scholar
(1981) Conversational organization. Interaction between speakers and hearers. New York, London.Google Scholar
Goodwin, M. H. & Goodwin, C.
(1986) Gesture and co-participation in the activity of searching for a word. Semiotica, 62(1/2). 51–75.Google Scholar
Gullberg, M. & Holmqvist, K.
(2006) What speakers do and what addressees look at. Visual attention to gestures in human interaction live and on video. Pragmatics & Cognition, 14, 53–82.DOI logoGoogle Scholar
Gullberg, M. & Kita, S.
(2009) Attention to speech-accompanying gestures: Eye movements and information uptake. Journal of Nonverbal Behaviour, 33, 251–277.DOI logoGoogle Scholar
Hanna, J. & Brennan, S.
(2007) Speakers’ eye gaze disambiguates referring expressions early during face-to-face conversation. Journal of Memory and Language, 57, 596–615.DOI logoGoogle Scholar
Hirvenkari, L., Ruusuvuori, J., Saarinen, V. -M., Kivioja, M. & Peräkylä, A.
(2013) Influence of turn-taking in a two-person conversation on the gaze of a viewer. PLoS ONE 8(8). DOI logoGoogle Scholar
Holler, J. & Kendrick, K.
(2015) Unaddressed participants’ gaze in multi-person interaction: Optimizing recipiency. Frontiers in Psychology, 6, 98. DOI logoGoogle Scholar
Jehoul, A., Brône, G., Feyaerts, K.
(2017/forthcoming). Gaze distribution during fillers: empirical data on the difference between Dutch ‘euh’ and ‘euhm’. In: P. Paggio et al. (eds.), Proceedings of the MMSYM Symposium ’16.Google Scholar
Jokinen, K.
(2010) Non-verbal signals for turn-taking & feedback. Proceedings of the 7th International Conference on Language Resources & Evaluation (LREC).Google Scholar
Jokinen, K., Nishida, M. & Yamamoto, S.
(2009) Eye gaze experiments for conversation monitoring. In: Proceedings of the 3rd International Universal Communication Symposium.DOI logoGoogle Scholar
Kendon, A.
(1967) Some functions of gaze-direction in social interaction. Acta Psychologica, 26, 22–63.DOI logoGoogle Scholar
Kendon A.
(1990) Conducting interaction: Patterns of behavior in focused encounters. Cambridge: Cambridge University Press, 1990.Google Scholar
Kendon, A.
(2004) Gesture: Visible action as utterance. Cambridge: Cambridge University Press.DOI logoGoogle Scholar
Kipp, M.
(2014) ANVIL: A universal video research tool. In: J. Durand, U. Gut & G. Kristofferson (Eds.), Handbook of corpus phonology (pp.420–443). Oxford: Oxford University Press.Google Scholar
Knight, D., Evans, D., Carter R. & Adolphs, S.
(2009) HeadTalk, HandTalk and the corpus: towards a framework for multi-modal, multi-media corpus development. Corpora, 4: 1–32.DOI logoGoogle Scholar
Knight, D., Adolphs, S., Tennent, P. & Carter, R.
(2008) The Nottingham multimodal corpus: a demonstration. In: Proceedings of the Sixth International Conference on Language Resources and Evaluation (LREC).
Kreysa, H. & Pickering, M.
(2011) Eye movements and dialogue. In S. Liversedge, D. Gilchrist & S. Everling (Eds.), The Oxford Handbook of Eye Movements (pp.943–959). Oxford: Oxford University Press.Google Scholar
Lachat, F., Conty, L., Hugueville, L. & George, N.
(2012) Gaze cueing effect in a face-to-face situation. Journal of Nonverbal Behavior, 36, 177–190.DOI logoGoogle Scholar
Linell, P.
(2009) Rethinking language, mind and world dialogically: Interactional and contextual theories of human sense-making. Charlotte, NC: Information Age Publishing.Google Scholar
Liversedge, D., Gilchrist, I. & Everling, S.
(Eds.) (2011) The Oxford handbook of eye movements. Oxford: Oxford University Press.DOI logoGoogle Scholar
Müller, C. et al.
(Eds.) (2013) Body – Language – Communication. An international handbook on multimodality in human interaction. Vol. 1. Berlin: Mouton de Gruyter.Google Scholar
(Eds.) (2014) Body – Language – Communication. An international handbook on multimodality in human interaction. Vol. 2. Berlin: Mouton de Gruyter.Google Scholar
Neider, M., Chen, X., Dickinson, C., Brennan, S. & Zelinsky, G.
(2010) Coordinating spatial referencing using shared gaze. Psychonomic Bulletin & Review, 17, 718–724.DOI logoGoogle Scholar
Novick, D. G., Hansen, B. & Ward, K.
(1996) Coordinating turn-taking with gaze. ICSLP, 3, 1888–1891.Google Scholar
Oben, B.
(2015) Modelling interactive alignment. A multimodal and temporal account. KU Leuven, unpublished PhD dissertation.Google Scholar
Oben, B. & Brône, G.
(2015) What you see is what you do. On the relationship between gaze and gesture in multimodal alignment. Language and Cognition 7 (4), 546–556.DOI logoGoogle Scholar
Oertel, C., Wlodarczak, M., Edlund, J., Wagner, P. & Gustafson, J.
(2012) Gaze patterns in turn-taking. Proceedings of Interspeech 2012.Google Scholar
Oloff, F.
(2012) Withdrawal from turns in overlap and participation. In P. Bergmann, J. Brenning, M. Pfeiffer & E. Reber (Eds.), Prosody and embodiment in interactional grammar (pp.207–236). Berlin/Boston: De Gruyter.DOI logoGoogle Scholar
(2013) Embodied withdrawal after overlap resolution. Journal of Pragmatics, 46, 139–156.DOI logoGoogle Scholar
Paggio, P., Allwood, J., Ahlsén, E., Jokinen, K. & Navarretta, C.
(2010) The NOMCO Multimodal Nordic Resource – Goals and characteristics. In: Proceedings of LREC 2010. Valletta, Malta
Pickering, M. & Garrod, S.
(2004) Toward a mechanistic psychology of dialogue. Behavioral and Brain Sciences, 27, 169–226.DOI logoGoogle Scholar
(2006) Alignment as basis for successful communication. Research on Language & Computation, 203–228.DOI logoGoogle Scholar
Pfeiffer, M., & Weiß, C.
(2017) Speaker gaze during descriptions and reenactments in storytelling. Paper presented at15th International Pragmatics Conference. Belfast, 16-21 July 2017.Google Scholar
Posner, M. I., Snyder, C. R., Davidson, B. J.
(1980) Attention and the detection of signals. Journal of Experimental Psychology, 109, 160–174.DOI logoGoogle Scholar
Postma-Nilsenova, M., Brunninkhuis, N. & Postma, E.
(2013) Eye gaze affects vocal intonation mimicry. Proceedings of the Cognitive Science Society, 35.Google Scholar
Rayner, K.
(1998) Eye movements in reading and information processing: 20 years of research. Psychological Bulletin, 85, 618–660.DOI logoGoogle Scholar
Richardson, D. & Dale, R.
(2005) Looking to understand: The coupling between speakers’ and listeners’ eye movements and its relationship to discourse comprehension. Cognitive Science, 29, 1045–1060.DOI logoGoogle Scholar
Richardson, D., Dale, R. & Kirkham, N.
(2007) The art of conversation is coordination. Common ground and the coupling of eye movements during dialogue. Psychological Science, 18, 407–413.DOI logoGoogle Scholar
Richardson, D., Dale, R. & Tomlinson, J.
(2009) Conversation, gaze coordination & beliefs about context. Cognitive Science, 33, 1468–1482.DOI logoGoogle Scholar
Rossano, F.
(2010) Questioning and responding in Italian. Journal of Pragmatics, 42, 2756–2771.DOI logoGoogle Scholar
(2012a) Gaze in conversation. In J. Sidnell & T. Stivers (Eds.), The Handbook of Conversation Analysis (pp.308–329). Chichester: Wiley-Blackwell.DOI logoGoogle Scholar
(2012b) Gaze behavior in face-to-face interaction. Nijmegen, The Netherlands: Max Planck Institute for Psycholinguistics unpublished dissertation.Google Scholar
Skantze, G., Hjalmarsson, A., & Oertel, C.
(2014) Turn-taking, feedback and joint attention in situated human-robot interaction. Speech Communication, 65, 50–66.DOI logoGoogle Scholar
Staudte, M. & Crocker, M.
(2011) Investigating joint attention mechanisms through spoken human-robot interaction. Cognition, 120, 268–291.DOI logoGoogle Scholar
Streeck, J.
(2009) Gesturecraft – The manufacture of meaning. Amsterdam/Philadelphia: John Benjamins.DOI logoGoogle Scholar
(2014) Mutual gaze and recognition: Revisiting Kendon’s ‘Gaze direction in two-person conversation’”, In M. Seyfeddinipur & M. Gullberg (Eds.), From gesture in conversation to visible action as utterance: Essays in honor of Adam Kendon (pp.35–56). Amsterdam: John Benjamins.Google Scholar
Stukenbrock, A.
(2015) Deixis in der face-to-face Interaktion. Berlin: Mouton De Gruyter.DOI logoGoogle Scholar
Sweetser, E. & Stec, K.
(2016) Maintaining multiple viewpoints with gaze. In B. Dancygier, L. Wei-Lun & A. Verhagen (Eds.), Viewpoint and the fabric of meaning (pp.237–258). Berlin: Mouton de Gruyter.DOI logoGoogle Scholar
Van Son, R., Wesseling, W., Sanders, E., Van Der Heuvel, H.
(2008) The IFADV corpus: A free dialog video corpus. In: Proceedings of the Sixth International rossanoConference on Language Resources and Evaluation (LREC).Google Scholar
Vertegaal, R.
(1999) The GAZE Groupware system: Mediating joint attention in multiparty communication and collaboration. In: M. G. Williams & M. W. Altom (Eds.), Proceedings of CHI’99 (pp.294–301). Pittsburgh: ACM.Google Scholar
Wang, Y. & Neff, M.
(2013) The Influence of Prosody on the Requirements for Gesture-Text Alignment. In R. Aylett, B. Krenn, C. Pelachaud & H. Shimodaira (Eds), Intelligent Virtual Agents. IVA 2013. Lecture Notes in Computer Science, vol 8108. Springer, Berlin, Heidelberg.DOI logoGoogle Scholar
Weiß, C. & Auer, P.
(2016) Das Blickverhalten des Rezipienten bei Sprecherhäsitationen: eine explorative Studie. Gesprächsforschung, 17, 132–167. [URL].Google Scholar
Wittenburg, P., Brugman, H., Russel, A., Klassmann, A. & Sloetjes, H.
(2006) ELAN: a Professional Framework for Multimodality Research. In: Proceedings of LREC 2006, Fifth International Conference on Language Resources and Evaluation.
Zima, E. & Brône, G.
(2015) Cognitive Linguistics and interactional discourse: time to enter into dialogue. Language and Cognition, 7, 485–498.DOI logoGoogle Scholar
Cited by

Cited by 4 other publications

Brône, Geert
2021. The multimodal negotiation of irony and humor in interaction. In Figurative Language – Intersubjectivity and Usage [Figurative Thought and Language, 11], ► pp. 109 ff. DOI logo
Li, Zhen, Michelle Mingyue Gu & Ye Zhu
2023. Identity and cultural intimacy of Central Asian students in a Hong Kong university. Globalisation, Societies and Education ► pp. 1 ff. DOI logo
Rasmussen, Gitte & Elisabeth Dalby Kristiansen
2022. The sociality of minimizing involvement in self-service shops in Denmark: Customers’ multi-modal practices of being, getting, and staying out of the way. Discourse & Communication 16:2 ► pp. 200 ff. DOI logo
Soipov, Jasur
2022. Some Reflections on Students Mobility and Labor Migration. In The Grass is Always Greener?, ► pp. 181 ff. DOI logo

This list is based on CrossRef data as of 22 april 2024. Please note that it may not be complete. Sources presented here have been supplied by the respective publishers. Any errors therein should be reported to them.