Audiovisual translation as orchestration of multimodal synergies: Expendability of interjections in intralingual film subtitling
Harbin Institute of Technology, Shenzhen
This study explores the coordination between subtitles and other semiotic resources in films, from within a systemic
functional semiotic framework. The paper focuses on the subtitling of interjections (e.g., ‘oh’, ‘wow’, ‘yay’) to examine how subtitles are
synthesized in films with respect to the presence of facial expressions and/or bodily gestures which might be perceived as conveying similar
meanings. The analysis of multiple versions of intralingual subtitles of two English-language films shows that the seemingly random omission
of soundtrack interjections from subtitles is patterned to a considerable degree – those interjections which are concurrent with
semiotically correlated actions were frequently omitted. The findings suggest that subtitling is a complex process involving synergy between
subtitles and other semiotic resources, which calls for interdisciplinary research integrating translation, multimodality, and
The study of audiovisual translation (AVT), which deals with multimodal texts, has naturally been closely aligned with multimodal approaches (e.g., Chaume 1997; Taylor 2003, 2004, 2013, 2016; Gambier 2006; Perego 2009; Burczynska 2012). This article contributes to the growing body of AVT research, and explores how subtitles are coordinated with other semiotic resources in films. This study focuses on interjections (e.g., ‘oh’, ‘wow’, ‘yay’, ‘aha’, ‘ouch’, ‘oops’, ‘hurray’), and examines the English intralingual subtitling of soundtrack interjections with respect to the presence of concurrent facial expressions and bodily gestures. For instance, it might be possible that where the visual shows a facial expression of raised eyebrows and widened eyes and mouth, the concurrent spoken interjection ‘wow’ might be considered expendable for subtitling. The assumption is that the meaning conveyed via the facial expression is analogous, or at least related, to the meaning conveyed via the utterance (in this case, ‘wow’), which, according to Gottlieb (1994, 273) may be regarded as a case of “intersemiotic redundancy.” This study interprets this hypothesis of intersemiotic redundancy from a systemic functional semiotic perspective, and suggests that intralingual film subtitling can be conceptualized as a process by which subtitlers orchestrate multimodal complementarities between their subtitles and other semiotic resources, such as facial expressions and bodily gestures.
DeMicco, Kirk, and Chris Sanders
2013The Croods. DreamWorks Animation. USA: 20th Century Fox, DVD.
1997 “Translating Non-verbal Information in Dubbing.” In Nonverbal Communication and Translation: New Perspectives and Challenges in Literature, Interpretation and the Media, edited by Fernando Poyatos, 315–326. Amsterdam: John Benjamins.
Cuenca, Maria Josep
2006 “Interjections and Pragmatic Errors in Dubbing.” Meta 51 (1): 20–35.
de Linde, Zoé
1995 “ ‘Read My Lips’ Subtitling Principles, Practices, and Problems.” Perspectives 3 (1): 9–20.
de Linde, Zoé, and Neil Kay
1999The Semiotics of Subtitling. Manchester: St. Jerome.
1994 “Subtitling: People Translating People.” In Teaching Translation and Interpreting 2: Insights, Aims, Visions. Papers from the Second Language International Conference, Elsinore, 1993, edited by Cay Dollerup and Annette Lindegaard, 261–274. Amsterdam: John Benjamins.
1994 “Relevance as a Factor in Subtitling Reductions.” In Teaching Translation and Interpreting 2: Insights, Aims, Visions. Papers from the Second Language International Conference. Elsinore, 1993, edited by Cay Dollerup and Annette Lindegaard, 245–251. Amsterdam: John Benjamins.
1998 “Six Subtitlers – Six Subtitling Texts.” In Unity in Diversity? Current Trends in Translation Studies, edited by Lynne Bowker, Michael Cronin, Dorothy Kenny, and Jennifer Pearson, 75–82. Manchester: St. Jerome.
2007 “The Multimodal Page: A Systemic Functional Exploration.” In New Directions in the Analysis of Multimodal Discourse, edited by Terry D. Royce and Wendy L. Bowcher, 1–62. Mahwah, NJ: Lawrence Erlbaum.
2006Making Comics: Storytelling Secrets of Comics, Manga and Graphic Novels. New York: Harper.
2001 “Features of Oral and Written Communication in Subtitling.” In (Multi) Media Translation: Concepts, Practices, and Research, edited by Yves Gambier and Henrik Gottlieb, 213–221. Amsterdam: John Benjamins.
2016Emotive Interjections in British English: A Corpus-based Study on Variation in Acquisition, Function and Usage. Amsterdam: John Benjamins.
2013 “Towards Interlingual Subtitling for the Deaf and the Hard of Hearing.” Perspectives 21 (1): 68–81.
2003 “Multimodal Transcription in the Analysis, Translation and Subtitling of Italian Films.” The Translator 9 (2): 191–205.
2004 “Multimodal Text Analysis and Subtitling.” In Perspectives on Multimodality, edited by Eija Ventola, Cassily Charles, and Martin Kaltenbacher, 153–172. Amsterdam: John Benjamins.
2013 “Multimodality and Audiovisual Translation.” In Handbook of Translation Studies, Volume 4, edited by Yves Gambier and Luc van Doorslaer, 98–104. Amsterdam: John Benjamins.
2016 “The Multimodal Approach in Audiovisual Translation.” In Audiovisual Translation: Theoretical and Methodological Challenges, edited by Yves Gambier and Sara Ramos Pinto, special issue of Target 28 (2): 222–236.
2011Multimodal Evaluation: Sense and Sensibility in Anthony Browne’s Picture Books. PhD diss. University of Sydney.
Wilkins, David P.
1992 “Interjections as Deictics.” Journal of Pragmatics 18 (2–3): 119–158.
1997 “Dubbing and the Nonverbal Dimension of Translation.” In Nonverbal Communication and Translation: New Perspectives and Challenges in Literature, Interpretation and the Media, edited by Fernando Poyatos, 327–342. Amsterdam: John Benjamins.