The importance of being relevant?A cognitive-pragmatic framework for conceptualising audiovisual translation
University of Surrey
Inspired by the belief that cognitive and pragmatic models of communication and discourse processing offer great potential for the study of Audiovisual Translation (AVT), this paper will review such models and discuss their contribution to conceptualising the three inter-related sub-processes underlying all forms of AVT: the comprehension of the multimodal discourse by the translator; the translation of selected elements of this discourse; and the comprehension of the newly formed multimodal discourse by the target audience. The focus will be on two models, Relevance Theory, which presents the most comprehensive pragmatic model of communication, and Mental Model Theory, which underlies cognitive models of discourse processing. The two approaches will be used to discuss and question common perceptions of AVT as being ‘constrained’ and ‘partial’ translation.
Many forms of audiovisual translation (AVT), especially subtitling, dubbing and voiceover, are primarily concerned with the translation of verbal language, be it from one language into another or from spoken into written language. Only subtitling for the deaf and hard of hearing (SDH) and audio description for the blind and partially sighted (AD) systematically require inter-modal transfer. However, given that the film dialogue or narrative, which is the object of translation in subtitling, dubbing and voiceover, is part of a multimodal discourse (i.e., a film or performance), these forms of AVT also require the translator to be aware of the other modes. At the same time, one of the challenges that all forms of AVT share is that the translator has control over only some elements of the multimodal discourse. The visual images are normally a ‘given’ and cannot be altered in the process of translation. Other challenges include the time and space limitations for the translated elements and requirements to achieve intermodal synchronicity (most prominently in dubbing), calling for appropriate strategies of information selection, condensation and/or omission. Although not exclusive to AVT, these challenges have led to AVT being conceptualised as ‘constrained translation’ (Bogucki 2004) and ‘partial translation’ (Benecke 2014). This paper argues that cognitive-pragmatic models of discourse processing enable us to re-evaluate these perceptions. It examines the potential of Mental Model Theory (Johnson-Laird 1983, 2006) and Relevance Theory (Sperber and Wilson 1995) for this purpose. Although these models have so far mainly been used to explain monomodal verbal discourse and monomodal translation, it will be shown that they can be applied to multimodal discourse as such and to AVT, and that the benefits of their application to AVT are wide-ranging.
Arnold, Derek, and David Whitney
2005 “Adaptation and Perceptual Binding in Sight and Sound.” In Fitting the Mind to the World, ed. by Colin Clifford, and Gillian Rhodes, 339–360. Oxford: Oxford University Press.
Bell, Victoria, and Philip Johnson-Laird
1998 “A Model Theory of Modal Reasoning.” Cognitive Science 22: 25–51.
2014Audiodeskription als partielle Translation — Modell und Methode. Berlin: LitVerlag.
2004 “The Constraint of Relevance in Subtitling.” JosTrans 1: 71–88.
2007a “Audio Description from a Discourse Perspective: A Socially Relevant Framework for Research and Training.” Linguistica Antverpiensia NS 6: 357–369.
2007b “Interpreting in Small-Group Bilingual Videoconferences: Challenges and Adaptation Processes.” Interpreting (9) 1: 21–46.
2011 “Creating Coherence in Audio Description.” Meta (56) 3: 645–662.
Brown, Gillian, and George Yule
1983Discourse Analysis. Cambridge: Cambridge University Press.
2012 “Implicatures in Film: Construal and Functions in Bridget Jones Romantic Comedies.” Journal of Pragmatics (44) 1: 30–53.
Diaz Cintas, Jorge, and Aline Remael
2007Subtitling. Manchester: St. Jerome.
2015From Multimodal Translation to ‘Multimodal Pragmatics’: Developing a Theoretical Framework for the Study of Multimodal Translation. PhD diss. University of Surrey.
Fels, Deborah, John Patrick Udo, Jonas Diamond, and Jeremy Diamond
2006 “Comparison of Alternative Narrative Approaches to Video Description for Animated Comedy.” Journal of Visual Impairment and Blindness (100) 5: 295–305.
2014 “Relevance Theory as a Model for Multimodal Communication.” In Visual Communication, ed. by David Machin, 51–70. Berlin: Mouton de Gruyter.
2009 “Challenges in Research on Audiovisual Translation.” In Translation Research Projects 2, ed. by Anthony Pym, and Alexander Perekrestenko, 24–33. Tarragona: Intercultural Studies Group.
2000Translation and Relevance. Cognition and Context. Manchester: St. Jerome.
2005 “On the Significance of the Cognitive Core of Translation.” The Translator (11) 1: 25–49.
2002Story Logic. Lincoln, NE: University of Nebraska Press.
2007Visual Communication: More Than Meets the Eye. Bristol: Intellect.
1983Mental Models. Cambridge, MA: Harvard University Press.
2006How We Reason. Oxford: Oxford University Press.
Kohn, Kurt, and Sylvia Kalina
1996 “The Strategic Dimension of Interpreting.” Meta (42) 1: 118–138.
1993 “Relevance as a Factor in Subtitling Reduction.” In Teaching Translation and Interpretation 2: Insights, Aims, Visions, ed. by Cay Dollerup, and Anne Lindegaard, 245–251. Amsterdam: John Benjamins.
1998 “Visual and Verbal Modes of Representation in Electronically Mediated Communication.” In Page to Screen, ed. by Illana Snyder, and Michael Joyce, 53–79. Sydney: Allen & Unwin.
2012 “Making Meaning in AVT: Eyetracking and Viewer Construction of Narrative.” Perspectives: Studies in Translatology (20) 1: 67–86.
2006 “Toward Critical Multimedia Literacy: Technology, Research, and Politics.” In International Handbook of Literacy and Technology, vol. 2, ed. by Michael McKenna, 2, 3–14. Mahwah, NJ: Erlbaum.
Martínez, Juan José Sierra
2010 “Approaching the Audio Description of Humour.” Entreculturas: revista de traducción y comunicación intercultural 2: 87–103.
Matamala, Anna, and Pilar Orero
2007 “Accessible Opera in Catalan: Opera for All.” In Media for All, ed. by Jorge Díaz Cintas, Pilar Orero, and Aline Remael, 201–214. Amsterdam: Rodopi.
2013Audio Description and Semiotics: The Translation of Films for Visually-Impaired Audiences. PhD diss. University of Surrey.
Orero, Pilar, and Anna Vilaró
2012 “Eyetracking Analysis of Minor Details in Films for Audio Description.” MonTI 4: 295–319.
1999Simultaneous Interpretation. A Cognitive-Pragmatic Approach. Amsterdam: John Benjamins.
Sperber, Dan, and Deirdre Wilson
1995Relevance. Communication and Cognition. 2nd ed. Oxford: Blackwell.
Van Dijk, Teun, and Walter Kintsch
1983Strategies of Discourse Comprehension. New York: Academic Press.
Vercauteren, Gert, and Aline Remael
2014 “Audio-Describing Spatio-Temporal Settings.” In Audio Description: New Perspectives Illustrated, ed. by Pilar Orero, Anna Matamala, and Anna Maszerowska, 61–80. Amsterdam: John Benjamins.
2008 “Inferring from Comics: a Multi-Stage Account.” Quaderns de Filologia. Estudis de Comunicació 3: 223–249.