Article published in:
Translation, Cognition & Behavior
Vol. 1:1 (2018) ► pp. 119–146

The effect of explanatory captions on the reception of foreign audiovisual products

A study drawing on eyetracking data and retrospective interviews

| Durham University
| Durham University
The present research triangulates questionnaire, retrospective interview and eyetracking data, aiming to investigate how Explanatory Captions (ECs) are received by different viewers with varied educational backgrounds, and whether or not the presence of ECs improves their understanding of the AV content. The results show that the provision of ECs, for a subtitled video in a foreign language, greatly increased positive cognitive effects on the viewers. Viewers tend to reduce time spent on viewing images, but invest additional processing effort on the ECs, although their allocation of processing effort on subtitles experienced little change. Furthermore, the eyetracking data suggest that most participants adopted a fixed reading pattern on subtitles and ECs when they appeared simultaneously, which could balance some of the negative impact of ECs on their viewing experience. The findings gained through this experimental research will provide some guidance and suggestions for subtitlers when preparing subtitles and ECs.
Keywords: Explanatory Caption, reception, Relevance Theory, eyetracking, retrospection
Article outline
  • 1.Introduction
  • 2.Theoretical framework
    • 2.1Relevance theory
    • 2.2Positive cognitive effect (PCEs) and processing effort
  • 3.Eye-tracking research on subtitle reception
  • 4.Methods
    • 4.1Participants
    • 4.2Stimuli
    • 4.3Eye movement recording and analysis
    • 4.4Questionnaire and retrospective interviews
    • 4.5Procedure
    • 4.6Quality assessment of eyetracking data
  • 5.Results
    • 5.1Positive Cognitive Effects (PCEs)
      • 5.1.1Self-evaluated PCEs
      • 5.1.2Tested PCEs
    • 5.2Processing efforts
      • 5.2.1Questionnaire and retrospection results
      • 5.2.2Eye-tracking results
  • 6.Discussion
  • 7.Conclusion
  • Notes
  • References


This article is currently available as a sample article.

Published online: 02 March 2018
https://doi.org/10.1075/tcb.00006.zhe
References
Bisson, Marie-Josée, Walter J. B. Van Heuven, Kathy Conklin, and Richard J. Tunney
2014 “Processing of Native and Foreign Language Subtitles in Films: An Eye Tracking Study.” Applied Psycholinguistics 35 (2): 399–418. CrossrefGoogle Scholar
Braun, Sabine
2016 “The Importance of Being Relevant? A Cognitive-Pragmatic Framework for Conceptualising Audiovisual Translation.” Target 28 (2): 302–313. CrossrefGoogle Scholar
Caffrey, Colm
2012 “Using an Eye–tracking Tool to Measure the Effects of Experimental Subtitling Procedures on Viewer Perception of Subtitled AV Content.” In Eye-tracking in Audiovisual Translation. Edited by E. Perego, 135–156. Roma: Aracne.Google Scholar
Dwyer, Tessa
2015 “From Subtitles to SMS: Eye Tracking, Texting and Sherlock .” Refractory: A Journal of Entertainment Media, 251. Accessed 11 Nov 2017: http://​refractory​.unimelb​.edu​.au​/2015​/02​/07​/dwyer/
d’Ydewalle, Géry, and Ingrid Gielen
1992 “Attention Allocation with Overlapping Sound, Image, and Text.” In Eye Movements and Visual Cognition: Scene Perception and Reading. Edited by K. Rayner, 415–427. New York: Springer. CrossrefGoogle Scholar
d’Ydewalle, Géry, and Wim De Bruycker
2007 “Eye Movements of Children and Adults while Reading Television Subtitles.” European Psychologist 12 (3): 196–205. CrossrefGoogle Scholar
d’Ydewalle, Géry, Johan van Rensbergen, and Joris Pollet
1987 “Reading a Message When the Same Message is Available Auditorily in Another Language: The Case of Subtitling.” In Eye Movements: From Physiology to Cognition. Edited by J. K. O’Regan and A. Lévy—Schoen, 313–321. Amsterdam: North—Holland. CrossrefGoogle Scholar
de Linde, Zoe, and Neil M. Kay
1999 The Semiotics of Subtitling. Manchester: St. Jerome.Google Scholar
Desilla, Louisa
2012 “Implicatures in Film: Construal and Functions in Bridget Jones Romantic Comedies.” Journal of Pragmatics 441: 30–53. CrossrefGoogle Scholar
Ghia, Elisa
2012 “The Impact of Translation Strategies on Subtitle Reading.” In Eye-tracking in Audiovisual Translation. Edited by E. Perego, 157–182. Roma: Aracne.Google Scholar
Hegarty, Mary
1992 “The Mechanics of Comprehension and Comprehension of Mechanics.” In Eye Movements and Visual Cognition: Scene Perception and Reading. Edited by K. Rayner, 428–443. New York: Springer. CrossrefGoogle Scholar
Hsiao, Chi-hua
2014 The Cultural Translation of U.S. Television Programs and Movies: Subtitle Groups as Cultural Brokers in China. PhD Thesis, University of California, Los Angeles.Google Scholar
Hvelplund, Kristian Tangsgaard
2011 Allocation of Cognitive Resources in Translation: An Eye-tracking and Key-logging Study. PhD thesis, Copenhagen Business School.Google Scholar
2014 “Eye Tracking and the Translation Process: Reflections on the Analysis and Interpretation of Eye-tracking Data. In Minding Translation. Edited by R. Muñoz, 201–223. MonTI, special issue 1. Alicante: Universidad de Alicante.Google Scholar
Kruger, Jan-Louis, Esté Hefer, and Gordon Matthew
2013 “Measuring the Impact of Subtitles on Cognitive Load: Eye Tracking and Dynamic Audiovisual Texts.” Proceedings of the 2013 Conference on Eye Tracking South Africa, 62–66. New York: ACM. CrossrefGoogle Scholar
Lautenbacher, Olli Philippe
2012 “From Still Pictures to Moving Pictures.” In Eye-tracking in Audiovisual Translation. Edited by E. Perego, 135–155. Roma: Aracne.Google Scholar
Moran, Siobhan
2012 “The Effect of Linguistic Variation on Subtitle Reception.” In Eye-tracking in Audiovisual Translation. Edited by E. Perego, 183–222. Roma: Aracne.Google Scholar
National Standard of the People’s Republic of China
2005 Target Text Quality Requirements for Translation Services. Beijing: Standards Press of China.Google Scholar
Perego, Elisa
2012 Eye Tracking in Audiovisual Translation. Roma: Aracne.Google Scholar
Rayner, Keith
1998 “Eye Movements in Reading and Information Processing: 20 Years of Research.” Psychological Bulletin 124 (3): 372–422. CrossrefGoogle Scholar
Rayner, Keith, and Alexander Pollatsek
1989 The Psychology of Reading. Englewood Cliffs, NJ: Prentice Hall.Google Scholar
Sperber, Dan, and Deirdre Wilson
1995 Relevance: Communication and Cognition. 2nd ed. Oxford: Blackwell.Google Scholar
Szarkowska, Agnieszka and Jan-Louis Kruger
2015 “Subtitles on the Moving Image: An Overview of Eye Tracking Studies.” Refractory: a Journal of Entertainment Media, 251. Accessed 11 Nov 2017: http://​refractory​.unimelb​.edu​.au​/2015​/02​/07​/kruger​-szarkowska​-krejtz/
Tannen, Deborah
1989 Talking Voices: Repetition, Dialogue and Imagery in Conversational Discourse. Cambridge: Cambridge University Press.Google Scholar
Thomas, Jenny
1995 Meaning in Interaction: An Introduction to Pragmatics. London: Longman.Google Scholar
Wilson, Deirdre, and Dan Sperber
2002 “Relevance Theory.” UCL Working papers in Linguistics 141: 249–287.Google Scholar
Yus Ramos, Francisco B.
2008 “Alterations of Relevance in Cyber—Media.” Universitas Psychologica 7 (3): 629–642.Google Scholar
Zheng, Binghan, and Xia Xiang
2014 “The Impact of Cultural Background Knowledge in the Processing of Metaphorical Expressions.” Translation and Interpreting Studies 9 (1): 5–24. CrossrefGoogle Scholar

Full-text

The effect of explanatory captions on the reception of foreign audiovisual products
Cited by

Cited by 4 other publications

Abdel Latif, Muhammad M. M.
2020.  In Translator and Interpreter Education Research [New Frontiers in Translation Studies, ], ► pp. 61 ff. Crossref logo
Abdel Latif, Muhammad M.M.
2019. Eye-tracking in recent L2 learner process research: A review of areas, issues, and methodological approaches. System 83 ► pp. 25 ff. Crossref logo
Sun, Sanjun & Kairong Xiao
2019. Chinese scholarship in Cognitive Translation Studies. Translation, Cognition & Behavior 2:1 ► pp. 125 ff. Crossref logo
Wu, Zhiwei & Zhuojia Chen
2021. A systematic reviewof experimental researchin audiovisual translation 1992–2020. Translation, Cognition & Behavior Crossref logo

This list is based on CrossRef data as of 07 february 2022. Please note that it may not be complete. Sources presented here have been supplied by the respective publishers. Any errors therein should be reported to them.