Publications
Publication details [#2479]
Publication type
Unpublished manuscript
Publication language
English
Keywords
Place, Publisher
Manoa, Hawaii
Abstract
Embodied approaches to language view meaning as fundamentally based on real-world perceptual and motor knowledge that an individual accrues through interactions with the world (Lakoff 1987, Langacker 1987). A unique aspect of this view is that understanding language results in internally imaging, or simulating, the content of the utterance (Bergen et al. 2004), Early computational work (Narayanan 1997) defined ways in which such simulations could be learned, represented, and used for natural language understanding. But until recently, little experimental work had evaluated the psychological reality of mental simulation during language processing. This paper presents results from a set of studies, demonstrating such a role for mental simulation.
The experimental work was based on by Richardson et al. (2003), and tested whether processing different sorts of spatial language makes use of corresponding parts of the visual system. Sentences encoding downwards motion like 'The bottle fell' contrast with others denoting upwards motion, like 'The ant climbed'. If language understanders perform modal mental simulations in order to understand sentences like these, then this entails activation of the corresponding areas of their visual fields. In this case, hearing sentences encoding upwards or downwards motion should selectively interfere with using the visual system for other purposes, since it cannot efficiently process multiple similar images (real and imagined) at the same time. An object categorization task tested this idea. Subjects first heard an up or down sentence, after which an object of varying shape appeared in the upper, lower, right, or left quadrant of a computer screen. Subjects categorized the shape of the object as quickly as possible (circle or square). Four types of intransitive sentences were tested: literal up or down sentences (like those above), metaphorical sentences using the same verbs (like 'The cost fell', and 'The temperature climbed'), abstract descriptions of quantity change (like 'The cost increased' versus 'The temperature decreased') and finally sentences in which the subject was strongly associated with upness or downness (like 'The grass glistened' versus 'The sky darkened'). We hypothesized that after up sentences (compared with down sentences), subjects would take longer to subsequently categorize an object in the upper part of the visual field, and vice versa.
Results showed that both types of concrete sentences (the ones encoding upwards versus downwards motion and the ones containing an up- or down-related subject noun phrase) yielded a significant interference effect on responses in the categorization task. The abstract sentences yielded no such effect, and there was a marginal effect in the case of the metaphorical sentences. This suggests that mental simulation is important to literal motion sentence comprehension, but may be less so for metaphorical and abstract language. More broadly, the results support the psychological reality of mental simulation during language understanding.
(Benjamin Bergen)