Chapter 5
On the role of gaze for successful and efficient communication
Speakers tend to fixate objects they are about to mention, while listeners inspect those objects that they believe to be intended referents of the speaker. These production- and comprehension-contingent gaze behaviors may form an integral part of the signal itself, making it inherently reciprocal. Here, we present work that has investigated the interplay of gaze and language and assessed the role of speaker gaze for language comprehension as well as the utility of listener gaze for an instruction giver. Both lines of research make use of artificial interaction partners which increases experimental control while maintaining a dynamic interactive setting. Thus, the reciprocal nature of situated dialogue becomes a tractable aspect in the enterprise of dealing with human (gaze) behavior.
Article outline
- 1.Introduction
- 2.The benefit of speaker gaze
- 3.Mechanisms underlying gaze benefit
- 4.The benefit of listener gaze
- 5.Discussion
-
References
References
Allopenna, P., Magnuson, J., & Tanenhaus, M.
(
1998)
Tracking the time course of spoken word recognition using eye movements: Evidence for continuous mapping models.
Journal of Memory and Language, 38, 419–439.
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Altmann, G., & Kamide, Y.
(
1999)
Incremental interpretation at verbs: restricting the domain of subsequent reference.
Cognition, 73, 247–264.
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Bayliss, A., Paul, M., Cannon, P., & Tipper, S.
(
2006)
Gaze cueing and affective judgments of objects: I like what you look at.
Psychonomic Bulletin & Review, 13, 1061–1066.
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Becchio, C., Bertone, C., & Castiello, U.
(
2008)
How the gaze of others influences object processing.
Trends in Cognitive Science, 12, 254–258.
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Brown-Schmidt, S., & Hanna, J. E.
(
2011)
Talking in another person’s shoes: Incremental perspective-taking in language processing.
Dialogue and Discourse, 2, 11–33.
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Brown-Schmidt, S., & Tanenhaus, M. K.
(
2008)
Real-time investigation of referential domains in unscripted conversation: A targeted language game approach.
Cognitive Science, 32(4), 643–684.
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Castiello, U.
(
2003)
Understanding Other People’s Actions: Intention and Attention.
Journal of Experimental Psychology, 29, 416–430.
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Clark, H. H., & Krych, M. A.
(
2004)
Speaking while monitoring addressees for understanding.
Journal of Memory and Language, 50, 62–81.
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Driver, J., Davis, G., Ricciardelli, P., Kidd, P., Maxwell, E., & Baron-Cohen, S.
(
1999)
Gaze Perception Triggers Reflexive Visuospatial Orienting.
Visual Cognition, 6(5), 509–540.
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Friesen, C., & Kingstone, A.
(
1998)
The eyes have it! Reflexive orienting is triggered by nonpredictive gaze.
Psychonomic Bulletin & Review, 5(3), 490–495.
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Garoufi, K., Staudte, M., Koller, A. & Crocker, M. W.
(
2016)
Exploiting listener gaze to improve situated communication in dynamic virtual environments.
Cognitive Science, 40:1671–1703.
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Gleitman, L. R., January, D., Nappa, R., & Trueswell, J. C.
(
2007)
On the give and take between event apprehension and utterance formulation.
Journal of Memory and Language, 57, 544–569.
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Griffin, Z. M., & Bock, K.
(
2000)
What the eyes say about speaking.
Psychological Science, 11, 274–279.
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Hanna, J., & Brennan, S.
(
2007)
Speakers’ eye gaze disambiguates referring expressions early during face-to-face conversation.
Journal of Memory and Language, 57(4), 596–615.
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Hanna, J. E., Tanenhaus, M. K., & Trueswell, J. C.
(
2003)
The effects of common ground and perspective on domains of referential interpretation.
Journal of Memory and Language, 49(1).
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Keysar, B., Barr, D. J., Balin, J. A., & Brauner, J. S.
(
2000)
Taking perspective in conversation: The role of mutual knowledge in comprehension.
Psychological Science, 11, 32–38.
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Koller, A., Garoufi, K., Staudte, M., & Crocker, M. W.
(
2012)
Enhancing referential success by tracking hearer gaze. In Proceedings of the 13th Annual Meeting of the Special Interest Group on Discourse and Dialogue. Seoul, South Korea.
Knoeferle, P. & Crocker, M. W.
(
2006)
The coordinated interplay of scene, utterance, and world knowledge: evidence from eye tracking.
Cognitive Science, 30(3):481–529.
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Knoeferle, P. & Crocker, M. W.
(
2007)
The influence of recent scene events on spoken comprehension: evidence from eye-movements.
Journal of Memory and Language (Special issue: Language-Vision Interaction); 57(2):519–543.
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Knoeferle, P., Pyykkönen-Klauck, P. & Crocker, M. W.
Langton, S. R. & Bruce, V.
(
1999)
Reflexive Visual Orienting in Response to the Social Attention of Others.
Visual Cognition, 6(5), 541–567.
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Meyer, A., Sleiderink, A., & Levelt, W.
(
1998)
Viewing and naming objects: Eye movements during noun phrase production.
Cognition, 66, B25–B33.
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Meltzoff, A. N., Brooks, R., Shon, A. P., & Rao, R. P.
(
2010)
Social robots are psychological agents for infants: A test of gaze following.
Neural Networks, 23(8–9), 966–972. (Social Cognition: From Babies to Robots)
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Metzing, C., & Brennan, S. E.
(
2003)
When conceptual pacts are broken: Partner-specific effects on the comprehension of referring expressions.
Journal of Memory and Language, 49(2), 201–213.
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Nappa, R. & Arnold, J.
(
2014)
The road to understanding is paved with the speaker’s intentions: Cues to the speaker’s attention and intentions affect pronoun comprehension.
Cognitive Psychology, 70, 58–81.
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Nappa, R., Wessel, A., McEldoon, K. L., Gleitman, L. R., & Trueswell, J. C.
(
2009)
Use of speaker’s gaze and syntax in verb learning.
Language Learning and Development, 5(4), 203–234.
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Nass, C., & Moon, Y.
(
2000)
Machines and mindlessness: Social responses to computers.
Journal of Social Issues, 56, 81–103.
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Rayner, K.
(
1998)
Eye movements in reading and information processing: 20 years of research.
Psychological Bulletin & Review, 124 (3), 372–422.
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Staudte, M., & Crocker, M. W.
(
2011)
Investigating joint attention mechanisms through spoken human-robot interaction.
Cognition, 120, 268–291.
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Staudte, M., Crocker, M. W., Heloir, A. & Michael Kipp
(
2014)
The influence of speaker gaze on listener comprehension: Contrasting visual versus intentional accounts.
Cognition, 133(1):317–328.
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Tanenhaus, M. K., Spivey-Knowlton, M., Eberhard, K., & Sedivy, J.
(
1995)
Integration of visual and linguistic information in spoken language comprehension.
Science, 268, 1632–1634.
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Tomlin, R. S.
(
1997)
Mapping conceptual representations into linguistic representations: The role of attention in grammar. In
J. Nuyts, &
E. Pederson (Eds.),
Language and conceptualization (pp.162–189), Cambridge: Cambridge University Press.
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Yoon, S. O., Koh, S., & Brown-Schmidt, S.
(
2012)
Influence of perspective and goals on reference production in conversation.
Psychonomic Bulletin and Review, 19, 699–707.
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Yu, C., & Smith, L. B.
(
2012)
Embodied attention and word learning by toddlers.
Cognition, 125, 244–262.
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Cited by
Cited by 1 other publications
Härkki, Tellervo
2023.
Mobile gaze tracking and an extended linkography for collaborative sketching and designing.
International Journal of Technology and Design Education 33:2
► pp. 379 ff.
![DOI logo](//benjamins.com/logos/doi-logo.svg)
This list is based on CrossRef data as of 26 june 2024. Please note that it may not be complete. Sources presented here have been supplied by the respective publishers.
Any errors therein should be reported to them.