Chapter published in:
Eye-tracking in Interaction: Studies on the role of eye gaze in dialogue
Edited by Geert Brône and Bert Oben
[Advances in Interaction Studies 10] 2018
► pp. 91–106

Chapter 5
On the role of gaze for successful and efficient communication

Speakers tend to fixate objects they are about to mention, while listeners inspect those objects that they believe to be intended referents of the speaker. These production- and comprehension-contingent gaze behaviors may form an integral part of the signal itself, making it inherently reciprocal. Here, we present work that has investigated the interplay of gaze and language and assessed the role of speaker gaze for language comprehension as well as the utility of listener gaze for an instruction giver. Both lines of research make use of artificial interaction partners which increases experimental control while maintaining a dynamic interactive setting. Thus, the reciprocal nature of situated dialogue becomes a tractable aspect in the enterprise of dealing with human (gaze) behavior.
Keywords: speaker gaze as communicative signal, listener gaze monitoring, real-time language comprehension, artificial agents
Article outline
  • 1.Introduction
  • 2.The benefit of speaker gaze
  • 3.Mechanisms underlying gaze benefit
  • 4.The benefit of listener gaze
  • 5.Discussion
  • References
Published online: 13 November 2018
https://doi.org/10.1075/ais.10.05sta
References
Allopenna, P., Magnuson, J., & Tanenhaus, M.
(1998) Tracking the time course of spoken word recognition using eye movements: Evidence for continuous mapping models. Journal of Memory and Language, 38, 419–439.CrossrefGoogle Scholar
Altmann, G., & Kamide, Y.
(1999) Incremental interpretation at verbs: restricting the domain of subsequent reference. Cognition, 73, 247–264.CrossrefGoogle Scholar
Bayliss, A., Paul, M., Cannon, P., & Tipper, S.
(2006) Gaze cueing and affective judgments of objects: I like what you look at. Psychonomic Bulletin & Review, 13, 1061–1066.CrossrefGoogle Scholar
Becchio, C., Bertone, C., & Castiello, U.
(2008) How the gaze of others influences object processing. Trends in Cognitive Science, 12, 254–258.CrossrefGoogle Scholar
Brown-Schmidt, S., & Hanna, J. E.
(2011) Talking in another person’s shoes: Incremental perspective-taking in language processing. Dialogue and Discourse, 2, 11–33. CrossrefGoogle Scholar
Brown-Schmidt, S., & Tanenhaus, M. K.
(2008) Real-time investigation of referential domains in unscripted conversation: A targeted language game approach. Cognitive Science, 32(4), 643–684.CrossrefGoogle Scholar
Castiello, U.
(2003) Understanding Other People’s Actions: Intention and Attention. Journal of Experimental Psychology, 29, 416–430.Google Scholar
Clark, H. H., & Krych, M. A.
(2004) Speaking while monitoring addressees for understanding. Journal of Memory and Language, 50, 62–81.CrossrefGoogle Scholar
Driver, J., Davis, G., Ricciardelli, P., Kidd, P., Maxwell, E., & Baron-Cohen, S.
(1999) Gaze Perception Triggers Reflexive Visuospatial Orienting. Visual Cognition, 6(5), 509–540.CrossrefGoogle Scholar
Friesen, C., & Kingstone, A.
(1998) The eyes have it! Reflexive orienting is triggered by nonpredictive gaze. Psychonomic Bulletin & Review, 5(3), 490–495.CrossrefGoogle Scholar
Garoufi, K., Staudte, M., Koller, A. & Crocker, M. W.
(2016) Exploiting listener gaze to improve situated communication in dynamic virtual environments. Cognitive Science, 40:1671–1703.CrossrefGoogle Scholar
Gleitman, L. R., January, D., Nappa, R., & Trueswell, J. C.
(2007) On the give and take between event apprehension and utterance formulation. Journal of Memory and Language, 57, 544–569.CrossrefGoogle Scholar
Griffin, Z. M., & Bock, K.
(2000) What the eyes say about speaking. Psychological Science, 11, 274–279.CrossrefGoogle Scholar
Hanna, J., & Brennan, S.
(2007) Speakers’ eye gaze disambiguates referring expressions early during face-to-face conversation. Journal of Memory and Language, 57(4), 596–615.CrossrefGoogle Scholar
Hanna, J. E., Tanenhaus, M. K., & Trueswell, J. C.
(2003) The effects of common ground and perspective on domains of referential interpretation. Journal of Memory and Language, 49(1).CrossrefGoogle Scholar
Keysar, B., Barr, D. J., Balin, J. A., & Brauner, J. S.
(2000) Taking perspective in conversation: The role of mutual knowledge in comprehension. Psychological Science, 11, 32–38.CrossrefGoogle Scholar
Koller, A., Garoufi, K., Staudte, M., & Crocker, M. W.
(2012) Enhancing referential success by tracking hearer gaze. In Proceedings of the 13th Annual Meeting of the Special Interest Group on Discourse and Dialogue. Seoul, South Korea.
Knoeferle, P. & Crocker, M. W.
(2006) The coordinated interplay of scene, utterance, and world knowledge: evidence from eye tracking. Cognitive Science, 30(3):481–529.CrossrefGoogle Scholar
(2007) The influence of recent scene events on spoken comprehension: evidence from eye-movements. Journal of Memory and Language (Special issue: Language-Vision Interaction); 57(2):519–543.CrossrefGoogle Scholar
Knoeferle, P., Pyykkönen-Klauck, P. & Crocker, M. W.
(Eds.) Visually Situated Language Comprehension. John Benjamins Publishing Company (2016).CrossrefGoogle Scholar
Langton, S. R. & Bruce, V.
(1999) Reflexive Visual Orienting in Response to the Social Attention of Others. Visual Cognition, 6(5), 541–567.CrossrefGoogle Scholar
Meyer, A., Sleiderink, A., & Levelt, W.
(1998) Viewing and naming objects: Eye movements during noun phrase production. Cognition, 66, B25–B33.CrossrefGoogle Scholar
Meltzoff, A. N., Brooks, R., Shon, A. P., & Rao, R. P.
(2010) Social robots are psychological agents for infants: A test of gaze following. Neural Networks, 23(8–9), 966–972. (Social Cognition: From Babies to Robots)CrossrefGoogle Scholar
Metzing, C., & Brennan, S. E.
(2003) When conceptual pacts are broken: Partner-specific effects on the comprehension of referring expressions. Journal of Memory and Language, 49(2), 201–213.CrossrefGoogle Scholar
Nappa, R. & Arnold, J.
(2014) The road to understanding is paved with the speaker’s intentions: Cues to the speaker’s attention and intentions affect pronoun comprehension. Cognitive Psychology, 70, 58–81.CrossrefGoogle Scholar
Nappa, R., Wessel, A., McEldoon, K. L., Gleitman, L. R., & Trueswell, J. C.
(2009) Use of speaker’s gaze and syntax in verb learning. Language Learning and Development, 5(4), 203–234.CrossrefGoogle Scholar
Nass, C., & Moon, Y.
(2000) Machines and mindlessness: Social responses to computers. Journal of Social Issues, 56, 81–103.CrossrefGoogle Scholar
Rayner, K.
(1998) Eye movements in reading and information processing: 20 years of research. Psychological Bulletin & Review, 124 (3), 372–422.CrossrefGoogle Scholar
Staudte, M., & Crocker, M. W.
(2011) Investigating joint attention mechanisms through spoken human-robot interaction. Cognition, 120, 268–291.CrossrefGoogle Scholar
Staudte, M., Crocker, M. W., Heloir, A. & Michael Kipp
(2014) The influence of speaker gaze on listener comprehension: Contrasting visual versus intentional accounts. Cognition, 133(1):317–328.CrossrefGoogle Scholar
Tanenhaus, M. K., Spivey-Knowlton, M., Eberhard, K., & Sedivy, J.
(1995) Integration of visual and linguistic information in spoken language comprehension. Science, 268, 1632–1634.CrossrefGoogle Scholar
Tomlin, R. S.
(1997) Mapping conceptual representations into linguistic representations: The role of attention in grammar. In J. Nuyts, & E. Pederson (Eds.), Language and conceptualization (pp.162–189), Cambridge: Cambridge University Press.CrossrefGoogle Scholar
Yoon, S. O., Koh, S., & Brown-Schmidt, S.
(2012) Influence of perspective and goals on reference production in conversation. Psychonomic Bulletin and Review, 19, 699–707.CrossrefGoogle Scholar
Yu, C., & Smith, L. B.
(2012) Embodied attention and word learning by toddlers. Cognition, 125, 244–262.CrossrefGoogle Scholar
Cited by

Cited by 1 other publications

Härkki, Tellervo
2022. Mobile gaze tracking and an extended linkography for collaborative sketching and designing. International Journal of Technology and Design Education Crossref logo

This list is based on CrossRef data as of 01 april 2022. Please note that it may not be complete. Sources presented here have been supplied by the respective publishers. Any errors therein should be reported to them.