Discourse, Vision, and Cognition
Author
While there is a growing body of psycholinguistic experimental research on mappings between language and vision on a word and sentence level, there are almost no studies on how speakers perceive, conceptualise and spontaneously describe a complex visual scene on higher levels of discourse. This book explores the relationship between language, eye movements and cognition, and brings together discourse analysis with cognitively oriented behavioral research. Based on the analysis of data drawn from spoken descriptive discourse, spontaneous conversation, and experimental investigations, this work offers a comprehensive picture of the dynamic natures of language, vision and mental imagery. Verbal and visual data, synchronised and correlated by means of a multimodal scoring method, are used as two windows to the mind to show how language and vision, in concert, can elucidate covert mental processes.
[Human Cognitive Processing, 23] 2008. xiii, 202 pp.
Publishing status: Available
© John Benjamins Publishing Company
Table of Contents
-
Preface | pp. xi–xiii
-
Chapter 1. Segmentation of spoken discourse | pp. 1–17
-
Chapter 2. Structure and content of spoken picture descriptions | pp. 19–38
-
Chapter 3. Description coherence and connection between foci | pp. 39–54
-
Chapter 4. Variations in picture description | pp. 55–77
-
Chapter 5. Multimodal sequential analysis and analytic tool | pp. 79–98
-
Chapter 6. Temporal correspondence between verbal and visual data | pp. 99–123
-
Chapter 7. Semantic correspondence between verbal and visual data | pp. 125–150
-
Chapter 8. Picture viewing, picture description and mental imagery | pp. 151–169
-
Chapter 9. Concluding chapter | pp. 171–178
-
-
Index | pp. 193–202
Cited by
Cited by 28 other publications
Ahmad, Gufran
2014. Analogical Thinking Induces Eye Movements in Scene Viewing. International Journal of Machine Learning and Computing 4:1 ► pp. 94 ff. 
Boeriis, Morten & Jana Holsanova
2012. Tracking visual segmentation: connecting semiotic and cognitive perspectives. Visual Communication 11:3 ► pp. 259 ff.
Bucher, Hans-Juergen & Philipp Niemann
2012. Visualizing science: the reception of powerpoint presentations. Visual Communication 11:3 ► pp. 283 ff.
Cialone, Claudia, Thora Tenbrink & Hugo J. Spiers
2018. Sculptors, Architects, and Painters Conceive of Depicted Spaces Differently. Cognitive Science 42:2 ► pp. 524 ff.
Cullipher, Steven & Hannah Sevian
2015. Atoms versus Bonds: How Students Look at Spectra. Journal of Chemical Education 92:12 ► pp. 1996 ff.
Friedrich, Maik, Seung Yong Lee, Paul Bates, Wayne Martin & Anja K. Faulhaber
2021. The influence of training level on manual flight in connection to performance, scan pattern, and task load. Cognition, Technology & Work 23:4 ► pp. 715 ff.
Gerstenberg, Annette
Hirvonen, Maija & Betta Saari
2022. Scripted or spontaneous? Two approaches to audio describing visual art in museums. Perspectives ► pp. 1 ff.
Holsanova, Jana
2012. New methods for studying visual communication and multimodal integration. Visual Communication 11:3 ► pp. 251 ff.
2016. A Cognitive Approach to Audio Description. In Researching Audio Description, ► pp. 49 ff.
2020. Uncovering scientific and multimodal literacy through audio description. Journal of Visual Literacy 39:3-4 ► pp. 132 ff.
Holsanova, Jana, Nils Holmberg & Kenneth Holmqvist
2009. Reading information graphics: The role of spatial contiguity and dual attentional guidance. Applied Cognitive Psychology 23:9 ► pp. 1215 ff.
Klein, Christoph, Juliane Betz, Martin Hirschbuehl, Caroline Fuchs, Barbara Schmiedtová, Martina Engelbrecht, Julia Mueller-Paul, Raphael Rosenberg & Kevin Paterson
2014. Describing Art – An Interdisciplinary Approach to the Effects of Speaking on Gaze Movements during the Beholding of Paintings. PLoS ONE 9:12 ► pp. e102439 ff.
Kruger, Jan-Louis
2012. Making meaning in AVT: eye tracking and viewer construction of narrative. Perspectives 20:1 ► pp. 67 ff.
Martinez, Santiago, John Isaacs, Fabiola Fernandez-Gutierrez, Daniel Gilmour & Ken Scott-Brown
2016. Building Bridges Between User and Designer: Co-creation, Immersion and Perspective Taking. In Advances in Design for Inclusion [Advances in Intelligent Systems and Computing, 500], ► pp. 117 ff.
Reinermann-Matatko, Anja
2015. Mapping a Local Budget Plan: Why and How?. In Modern Trends in Cartography [Lecture Notes in Geoinformation and Cartography, ], ► pp. 467 ff.
Sandgren, Håkan
2010. The Intermediality of Field Guides: Notes Towards a Theory. In Media Borders, Multimodality and Intermediality, ► pp. 111 ff.
Soler Gallego, Silvia
2018. Audio descriptive guides in art museums. Translation and Interpreting Studies 13:2 ► pp. 230 ff.
Taboada, Maite & Christopher Habel
2013. Rhetorical relations in multimodal documents. Discourse Studies 15:1 ► pp. 65 ff.
Taylor, Christopher
2015. Language as Access: Transposition and Translation of Audiovisual Texts as a Vehicle of Meaning and a Gateway to Understanding. In Meaning Making in Text, ► pp. 170 ff.
Tenbrink, Thora
2012. Relevance in Spatial Navigation and Communication. In Spatial Cognition VIII [Lecture Notes in Computer Science, 7463], ► pp. 358 ff.
2015. Cognitive Discourse Analysis: accessing cognitive representations and processes through language data. Language and Cognition 7:1 ► pp. 98 ff.
2020. Cognitive Discourse Analysis,
van Gog, Tamara & Halszka Jarodzka
2013. Eye Tracking as a Tool to Study and Enhance Cognitive and Metacognitive Processes in Computer-Based Learning Environments. In International Handbook of Metacognition and Learning Technologies [Springer International Handbooks of Education, 28], ► pp. 143 ff.
Wang, Jiayu
2014. Criticising images: critical discourse analysis of visual semiosis in picture news. Critical Arts 28:2 ► pp. 264 ff.
Waugh, Linda R., Theresa Catalano, Khaled Al Masaeed, Tom Hong Do & Paul G. Renigar
2016. Critical Discourse Analysis: Definition, Approaches, Relation to Pragmatics, Critique, and Trends. In Interdisciplinary Studies in Pragmatics, Culture and Society [Perspectives in Pragmatics, Philosophy & Psychology, 4], ► pp. 71 ff.
Zhao, Qingli
2016. Book Review: Multimodality, Cognition, and Experimental Literature, by Alison Gibbons. Visual Communication 15:1 ► pp. 125 ff.
[no author supplied]
2008. PUBLICATIONS RECEIVED. World Englishes 27:3-4 ► pp. 566 ff.
This list is based on CrossRef data as of 6 march 2023. Please note that it may not be complete. Sources presented here have been supplied by the respective publishers. Any errors therein should be reported to them.
Subjects & Metadata
Psychology
BIC Subject: JMR – Cognition & cognitive psychology
BISAC Subject: PSY008000 – PSYCHOLOGY / Cognitive Psychology & Cognition