Chapter published in:
Eye-tracking in Interaction: Studies on the role of eye gaze in dialogue
Edited by Geert Brône and Bert Oben
[Advances in Interaction Studies 10] 2018
► pp. 91–106

Chapter 5
On the role of gaze for successful and efficient communication

Speakers tend to fixate objects they are about to mention, while listeners inspect those objects that they believe to be intended referents of the speaker. These production- and comprehension-contingent gaze behaviors may form an integral part of the signal itself, making it inherently reciprocal. Here, we present work that has investigated the interplay of gaze and language and assessed the role of speaker gaze for language comprehension as well as the utility of listener gaze for an instruction giver. Both lines of research make use of artificial interaction partners which increases experimental control while maintaining a dynamic interactive setting. Thus, the reciprocal nature of situated dialogue becomes a tractable aspect in the enterprise of dealing with human (gaze) behavior.

Keywords: speaker gaze as communicative signal, listener gaze monitoring, real-time language comprehension, artificial agents
Article outline
  • 1.Introduction
  • 2.The benefit of speaker gaze
  • 3.Mechanisms underlying gaze benefit
  • 4.The benefit of listener gaze
  • 5.Discussion
  • References
Published online: 13 November 2018
https://doi.org/10.1075/ais.10.05sta
References
Allopenna, P., Magnuson, J., & Tanenhaus, M.
(1998) Tracking the time course of spoken word recognition using eye movements: Evidence for continuous mapping models. Journal of Memory and Language, 38, 419–439.DOI logoGoogle Scholar
Altmann, G., & Kamide, Y.
(1999) Incremental interpretation at verbs: restricting the domain of subsequent reference. Cognition, 73, 247–264.DOI logoGoogle Scholar
Bayliss, A., Paul, M., Cannon, P., & Tipper, S.
(2006) Gaze cueing and affective judgments of objects: I like what you look at. Psychonomic Bulletin & Review, 13, 1061–1066.DOI logoGoogle Scholar
Becchio, C., Bertone, C., & Castiello, U.
(2008) How the gaze of others influences object processing. Trends in Cognitive Science, 12, 254–258.DOI logoGoogle Scholar
Brown-Schmidt, S., & Hanna, J. E.
(2011) Talking in another person’s shoes: Incremental perspective-taking in language processing. Dialogue and Discourse, 2, 11–33. DOI logoGoogle Scholar
Brown-Schmidt, S., & Tanenhaus, M. K.
(2008) Real-time investigation of referential domains in unscripted conversation: A targeted language game approach. Cognitive Science, 32(4), 643–684.DOI logoGoogle Scholar
Castiello, U.
(2003) Understanding Other People’s Actions: Intention and Attention. Journal of Experimental Psychology, 29, 416–430.Google Scholar
Clark, H. H., & Krych, M. A.
(2004) Speaking while monitoring addressees for understanding. Journal of Memory and Language, 50, 62–81.DOI logoGoogle Scholar
Driver, J., Davis, G., Ricciardelli, P., Kidd, P., Maxwell, E., & Baron-Cohen, S.
(1999) Gaze Perception Triggers Reflexive Visuospatial Orienting. Visual Cognition, 6(5), 509–540.DOI logoGoogle Scholar
Friesen, C., & Kingstone, A.
(1998) The eyes have it! Reflexive orienting is triggered by nonpredictive gaze. Psychonomic Bulletin & Review, 5(3), 490–495.DOI logoGoogle Scholar
Garoufi, K., Staudte, M., Koller, A. & Crocker, M. W.
(2016) Exploiting listener gaze to improve situated communication in dynamic virtual environments. Cognitive Science, 40:1671–1703.DOI logoGoogle Scholar
Gleitman, L. R., January, D., Nappa, R., & Trueswell, J. C.
(2007) On the give and take between event apprehension and utterance formulation. Journal of Memory and Language, 57, 544–569.DOI logoGoogle Scholar
Griffin, Z. M., & Bock, K.
(2000) What the eyes say about speaking. Psychological Science, 11, 274–279.DOI logoGoogle Scholar
Hanna, J., & Brennan, S.
(2007) Speakers’ eye gaze disambiguates referring expressions early during face-to-face conversation. Journal of Memory and Language, 57(4), 596–615.DOI logoGoogle Scholar
Hanna, J. E., Tanenhaus, M. K., & Trueswell, J. C.
(2003) The effects of common ground and perspective on domains of referential interpretation. Journal of Memory and Language, 49(1).DOI logoGoogle Scholar
Keysar, B., Barr, D. J., Balin, J. A., & Brauner, J. S.
(2000) Taking perspective in conversation: The role of mutual knowledge in comprehension. Psychological Science, 11, 32–38.DOI logoGoogle Scholar
Koller, A., Garoufi, K., Staudte, M., & Crocker, M. W.
(2012) Enhancing referential success by tracking hearer gaze. In Proceedings of the 13th Annual Meeting of the Special Interest Group on Discourse and Dialogue. Seoul, South Korea.
Knoeferle, P. & Crocker, M. W.
(2006) The coordinated interplay of scene, utterance, and world knowledge: evidence from eye tracking. Cognitive Science, 30(3):481–529.DOI logoGoogle Scholar
(2007) The influence of recent scene events on spoken comprehension: evidence from eye-movements. Journal of Memory and Language (Special issue: Language-Vision Interaction); 57(2):519–543.DOI logoGoogle Scholar
Knoeferle, P., Pyykkönen-Klauck, P. & Crocker, M. W.
(Eds.) Visually Situated Language Comprehension. John Benjamins Publishing Company (2016).DOI logoGoogle Scholar
Langton, S. R. & Bruce, V.
(1999) Reflexive Visual Orienting in Response to the Social Attention of Others. Visual Cognition, 6(5), 541–567.DOI logoGoogle Scholar
Meyer, A., Sleiderink, A., & Levelt, W.
(1998) Viewing and naming objects: Eye movements during noun phrase production. Cognition, 66, B25–B33.DOI logoGoogle Scholar
Meltzoff, A. N., Brooks, R., Shon, A. P., & Rao, R. P.
(2010) Social robots are psychological agents for infants: A test of gaze following. Neural Networks, 23(8–9), 966–972. (Social Cognition: From Babies to Robots)DOI logoGoogle Scholar
Metzing, C., & Brennan, S. E.
(2003) When conceptual pacts are broken: Partner-specific effects on the comprehension of referring expressions. Journal of Memory and Language, 49(2), 201–213.DOI logoGoogle Scholar
Nappa, R. & Arnold, J.
(2014) The road to understanding is paved with the speaker’s intentions: Cues to the speaker’s attention and intentions affect pronoun comprehension. Cognitive Psychology, 70, 58–81.DOI logoGoogle Scholar
Nappa, R., Wessel, A., McEldoon, K. L., Gleitman, L. R., & Trueswell, J. C.
(2009) Use of speaker’s gaze and syntax in verb learning. Language Learning and Development, 5(4), 203–234.DOI logoGoogle Scholar
Nass, C., & Moon, Y.
(2000) Machines and mindlessness: Social responses to computers. Journal of Social Issues, 56, 81–103.DOI logoGoogle Scholar
Rayner, K.
(1998) Eye movements in reading and information processing: 20 years of research. Psychological Bulletin & Review, 124 (3), 372–422.DOI logoGoogle Scholar
Staudte, M., & Crocker, M. W.
(2011) Investigating joint attention mechanisms through spoken human-robot interaction. Cognition, 120, 268–291.DOI logoGoogle Scholar
Staudte, M., Crocker, M. W., Heloir, A. & Michael Kipp
(2014) The influence of speaker gaze on listener comprehension: Contrasting visual versus intentional accounts. Cognition, 133(1):317–328.DOI logoGoogle Scholar
Tanenhaus, M. K., Spivey-Knowlton, M., Eberhard, K., & Sedivy, J.
(1995) Integration of visual and linguistic information in spoken language comprehension. Science, 268, 1632–1634.DOI logoGoogle Scholar
Tomlin, R. S.
(1997) Mapping conceptual representations into linguistic representations: The role of attention in grammar. In J. Nuyts, & E. Pederson (Eds.), Language and conceptualization (pp.162–189), Cambridge: Cambridge University Press.DOI logoGoogle Scholar
Yoon, S. O., Koh, S., & Brown-Schmidt, S.
(2012) Influence of perspective and goals on reference production in conversation. Psychonomic Bulletin and Review, 19, 699–707.DOI logoGoogle Scholar
Yu, C., & Smith, L. B.
(2012) Embodied attention and word learning by toddlers. Cognition, 125, 244–262.DOI logoGoogle Scholar
Cited by

Cited by 1 other publications

Härkki, Tellervo
2022. Mobile gaze tracking and an extended linkography for collaborative sketching and designing. International Journal of Technology and Design Education Crossref logo

This list is based on CrossRef data as of 05 january 2023. Please note that it may not be complete. Sources presented here have been supplied by the respective publishers. Any errors therein should be reported to them.