Multimodality and audiovisual translation

Christopher John Taylor
University of Trieste
Table of contents

Multimodality or “the use of several semiotic modes in the design of a semiotic product or event” (Kress & Van Leeuwen 2001: 20) is not a new field of study in that everything is to some extent multimodal, but in the modern world, archetypal multimodal texts such as films, television programmes and websites, have greatly broadened the scope of such studies. The term ‘semiotic mode’ has been variously labelled as ‘semiotic modality’, ‘semiotic resource’, etc. by different authors, but in this article for the sake of clarity the term mode will be used throughout. The ever-growing importance of multimodality concerns practically all disciplines and, by extension, the field of translation. In particular, research in Audiovisual translation would seem by its very nature to be dependent on multimodality studies, and a limited number of scholars have in fact ventured into this field (Baldry 2000; Taylor 2003, 2004; Gambier 2006; Perego 2009; Burczynska 2012). The opposite perspective, that is the role of translation in multimodality studies seems, on the other hand, to have attracted little or no interest. The Routledge Handbook of Multimodal Analysis (2009), an authoritative 340-page volume on the state of the art of multimodality, does not even mention translation in the index. It would therefore seem to fall on audiovisual translators themselves to seek and find the keys to unlock the potential of multimodal analysis in the service of the translation of screen products.

Full-text access is restricted to subscribers. Log in to obtain additional credentials. For subscription information see Subscription & Price.

References

Araújo, Vera L.S
2007 “Subtitling for the deaf and hard-of-hearing in Brazil.” In Media for All: Subtitling for the Deaf, Audio Description and Sign Language, Jorge Diaz Cintas, Pilar Orero & Aline Remael (eds), 99–107. Kenilworth, NJ: Rodopi.Google Scholar
Baldry, Anthony
(ed.) 2000 Multimodality and Multimediality in the distance learning age. Campobasso: Palladino Editore.Google Scholar
2005 A Multimodal Approach to Text Studies in English. Campobasso: Palladino Editore.Google Scholar
Baldry, Anthony & Thibault, Paul
2006 Multimodal Transcription and Text Analysis. London: Equinox.Google Scholar
Burczynska, Paulina
2012 Multimodality and Audiovisual Translation: Subtitling Humo. Saarbrücken: Lambert Academic Publishing.Google Scholar
Gambier, Yves
2006 “Multimodality and audiovisual translation.” In Audiovisual Translation Scenarios: Proceedings of the Marie Curie Euroconferences MuTra: Audiovisual Translation Scenarios, Mary Carroll, Heidrun Gerzymisch-Arbogast & Sandra Nauert (eds). Copenhagen 1–5 May 2006.  TSBGoogle Scholar
Halliday, Michael A.K
1978 Language as Social Semiotic, the Social Interpretation of Language and Meaning. London: Edward Arnold.Google Scholar
1994 An Introduction to Functional Grammar [2nd edition]. London: Edward Arnold.Google Scholar
Jewitt, Carey
(ed.) 2009 The Routledge Handbook of Multimodal Analysis. Abingdon: Routledge.Google Scholar
Jones, Rodney H
2009 “Technology and sites of display.” In The Routledge Handbook of Multimodal Analysis, Carey Jewitt (ed.) 114–126. Abingdon: Routledge.Google Scholar
Kress, Gunther
2009 Multimodality. A Social Semiotic Approach to Communication. London: -Routledge Falmer.Google Scholar
Kress, Gunther & Leeuwen, Theo Van
2001 Multimodal Discourse: the Modes and Media of Contemporary Communication. London: Edward Arnold.  BoPGoogle Scholar
2006 Reading Images: The Grammar of Visual Design. -London: Routledge Falmer.  BoPGoogle Scholar
Martinec, Radan & Salway, Andrew
2005 “A system for image-text relations in new (and old) media.” Visual Communication 4 (3): 337–371. DOI logoGoogle Scholar
O’Hagan, Minako
2009 “Putting pleasure first: Localizing Japanese video games.” The Translator 22 (1): 147–166. DOI logo  TSBGoogle Scholar
O’Halloran, Kay
2004 “Visual semiosis in films.” In Multimodal Discourse Analysis: Systemic Functional Perspectives, Kay O’Halloran (ed.), 109–130. London: Continuum.Google Scholar
O’Toole, Michael
1994 The Language of Displayed Art. London: Leicester University Press.Google Scholar
Perego, Elisa
2009 “The codification of non-verbal information in subtitled texts.” In New Trends in Audiovisual Translation, Jorge D. Cintas (ed.),. 58–69. Bristol: Multilingual Matters.  TSBGoogle Scholar
Taylor, Christopher
2003 “Multimodal transcription in the analysis: Translation and subtitling of Italian films.” The Translator (9) 2: 191–206. DOI logo  TSBGoogle Scholar
2004 “Multimodal text analysis and subtitling.” In Perspectives on Multimodality, Eija Ventola, Cassily Charles & Martin Kaltenbacher (eds), 153–172. Amsterdam: John Benjamins. DOI logoGoogle Scholar
Thibault, Paul
2000 “The multimodal transcription of a television advertisement: Theory and practice.” In Multimodality and Multimediality in the Distance Learning Age, Anthony Baldry (ed.), 311–385. Campobasso: Palladino Editore.Google Scholar
Tomaszkiewicz, Teresa
2010 “Areas of untranslatability in audiovisual transfers.” In Perspectives on Audiovisual Translation, Łukasz Bogucki & Krzysztof Kredens (eds), 93–106. Frankfurt: Peter Lang.Google Scholar

Film reference

Tarantino, Quentin
2009 Inglorious Bastards.Google Scholar