Cover not available
Article published In:
Translation, Cognition & Behavior
Vol. 1:1 (2018) ► pp.119–146

The effect of explanatory captions on the reception of foreign audiovisual products

A study drawing on eyetracking data and retrospective interviews

 | Durham University
 | Durham University
The present research triangulates questionnaire, retrospective interview and eyetracking data, aiming to investigate how Explanatory Captions (ECs) are received by different viewers with varied educational backgrounds, and whether or not the presence of ECs improves their understanding of the AV content. The results show that the provision of ECs, for a subtitled video in a foreign language, greatly increased positive cognitive effects on the viewers. Viewers tend to reduce time spent on viewing images, but invest additional processing effort on the ECs, although their allocation of processing effort on subtitles experienced little change. Furthermore, the eyetracking data suggest that most participants adopted a fixed reading pattern on subtitles and ECs when they appeared simultaneously, which could balance some of the negative impact of ECs on their viewing experience. The findings gained through this experimental research will provide some guidance and suggestions for subtitlers when preparing subtitles and ECs.
Keywords: Explanatory Caption, reception, Relevance Theory, eyetracking, retrospection
Article outline
  • 1.Introduction
  • 2.Theoretical framework
    • 2.1Relevance theory
    • 2.2Positive cognitive effect (PCEs) and processing effort
  • 3.Eye-tracking research on subtitle reception
  • 4.Methods
    • 4.1Participants
    • 4.2Stimuli
    • 4.3Eye movement recording and analysis
    • 4.4Questionnaire and retrospective interviews
    • 4.5Procedure
    • 4.6Quality assessment of eyetracking data
  • 5.Results
    • 5.1Positive Cognitive Effects (PCEs)
      • 5.1.1Self-evaluated PCEs
      • 5.1.2Tested PCEs
    • 5.2Processing efforts
      • 5.2.1Questionnaire and retrospection results
      • 5.2.2Eye-tracking results
  • 6.Discussion
  • 7.Conclusion
  • Notes
  • References


This article is currently available as a sample article.

Published online: 2 March 2018
DOI logo
https://doi.org/10.1075/tcb.00006.zhe
References (28)
References
Bisson, Marie-Josée, Walter J. B. Van Heuven, Kathy Conklin, and Richard J. Tunney. 2014. “Processing of Native and Foreign Language Subtitles in Films: An Eye Tracking Study.” Applied Psycholinguistics 35 (2): 399–418. DOI logoGoogle Scholar
Caffrey, Colm. 2012. “Using an Eye–tracking Tool to Measure the Effects of Experimental Subtitling Procedures on Viewer Perception of Subtitled AV Content.” In Eye-tracking in Audiovisual Translation. Edited by E. Perego, 135–156. Roma: Aracne.Google Scholar
Dwyer, Tessa. 2015. “From Subtitles to SMS: Eye Tracking, Texting and Sherlock .” Refractory: A Journal of Entertainment Media, 251. Accessed 11 Nov 2017: [URL]
d’Ydewalle, Géry, and Ingrid Gielen. 1992. “Attention Allocation with Overlapping Sound, Image, and Text.” In Eye Movements and Visual Cognition: Scene Perception and Reading. Edited by K. Rayner, 415–427. New York: Springer. DOI logoGoogle Scholar
d’Ydewalle, Géry, and Wim De Bruycker. 2007. “Eye Movements of Children and Adults while Reading Television Subtitles.” European Psychologist 12 (3): 196–205. DOI logoGoogle Scholar
d’Ydewalle, Géry, Johan van Rensbergen, and Joris Pollet. 1987. “Reading a Message When the Same Message is Available Auditorily in Another Language: The Case of Subtitling.” In Eye Movements: From Physiology to Cognition. Edited by J. K. O’Regan and A. Lévy—Schoen, 313–321. Amsterdam: North—Holland. DOI logoGoogle Scholar
de Linde, Zoe, and Neil M. Kay. 1999. The Semiotics of Subtitling. Manchester: St. Jerome.Google Scholar
Desilla, Louisa. 2012. “Implicatures in Film: Construal and Functions in Bridget Jones Romantic Comedies.” Journal of Pragmatics 441: 30–53. DOI logoGoogle Scholar
Ghia, Elisa. 2012. “The Impact of Translation Strategies on Subtitle Reading.” In Eye-tracking in Audiovisual Translation. Edited by E. Perego, 157–182. Roma: Aracne.Google Scholar
Hegarty, Mary. 1992. “The Mechanics of Comprehension and Comprehension of Mechanics.” In Eye Movements and Visual Cognition: Scene Perception and Reading. Edited by K. Rayner, 428–443. New York: Springer. DOI logoGoogle Scholar
Hsiao, Chi-hua. 2014. The Cultural Translation of U.S. Television Programs and Movies: Subtitle Groups as Cultural Brokers in China. PhD Thesis, University of California, Los Angeles.Google Scholar
Hvelplund, Kristian Tangsgaard. 2011. Allocation of Cognitive Resources in Translation: An Eye-tracking and Key-logging Study. PhD thesis, Copenhagen Business School.Google Scholar
. 2014. “Eye Tracking and the Translation Process: Reflections on the Analysis and Interpretation of Eye-tracking Data. In Minding Translation. Edited by R. Muñoz, 201–223. MonTI, special issue 1. Alicante: Universidad de Alicante.Google Scholar
Kruger, Jan-Louis, Esté Hefer, and Gordon Matthew. 2013. “Measuring the Impact of Subtitles on Cognitive Load: Eye Tracking and Dynamic Audiovisual Texts.” Proceedings of the 2013 Conference on Eye Tracking South Africa, 62–66. New York: ACM. DOI logoGoogle Scholar
Lautenbacher, Olli Philippe. 2012. “From Still Pictures to Moving Pictures.” In Eye-tracking in Audiovisual Translation. Edited by E. Perego, 135–155. Roma: Aracne.Google Scholar
Moran, Siobhan. 2012. “The Effect of Linguistic Variation on Subtitle Reception.” In Eye-tracking in Audiovisual Translation. Edited by E. Perego, 183–222. Roma: Aracne.Google Scholar
National Standard of the People’s Republic of China. 2005. Target Text Quality Requirements for Translation Services. Beijing: Standards Press of China.Google Scholar
Perego, Elisa. 2012. Eye Tracking in Audiovisual Translation. Roma: Aracne.Google Scholar
Rayner, Keith. 1998. “Eye Movements in Reading and Information Processing: 20 Years of Research.” Psychological Bulletin 124 (3): 372–422. DOI logoGoogle Scholar
Rayner, Keith, and Alexander Pollatsek. 1989. The Psychology of Reading. Englewood Cliffs, NJ: Prentice Hall.Google Scholar
Sperber, Dan, and Deirdre Wilson. 1995. Relevance: Communication and Cognition. 2nd ed. Oxford: Blackwell.Google Scholar
Szarkowska, Agnieszka and Jan-Louis Kruger. 2015. “Subtitles on the Moving Image: An Overview of Eye Tracking Studies.” Refractory: a Journal of Entertainment Media, 251. Accessed 11 Nov 2017: [URL]
Tannen, Deborah. 1989. Talking Voices: Repetition, Dialogue and Imagery in Conversational Discourse. Cambridge: Cambridge University Press.Google Scholar
Thomas, Jenny. 1995. Meaning in Interaction: An Introduction to Pragmatics. London: Longman.Google Scholar
Wilson, Deirdre, and Dan Sperber. 2002. “Relevance Theory.” UCL Working papers in Linguistics 141: 249–287.Google Scholar
Yus Ramos, Francisco B. 2008. “Alterations of Relevance in Cyber—Media.” Universitas Psychologica 7 (3): 629–642.Google Scholar
Zheng, Binghan, and Xia Xiang. 2014. “The Impact of Cultural Background Knowledge in the Processing of Metaphorical Expressions.” Translation and Interpreting Studies 9 (1): 5–24. DOI logoGoogle Scholar
Cited by (5)

Cited by five other publications

Hu, Bei
2023. Chapter 10. Flowing to the reception side. In Translation Flows [Benjamins Translation Library, 163], ► pp. 183 ff. DOI logo
Wu, Zhiwei & Zhuojia Chen
2021. A systematic review of experimental research in audiovisual translation 1992–2020. Translation, Cognition & Behavior 4:2 ► pp. 281 ff. DOI logo
Abdel Latif, Muhammad M. M.
2020. Translation and Interpreting Assessment Research. In Translator and Interpreter Education Research [New Frontiers in Translation Studies, ], ► pp. 61 ff. DOI logo
Abdel Latif, Muhammad M.M.
2019. Eye-tracking in recent L2 learner process research: A review of areas, issues, and methodological approaches. System 83 ► pp. 25 ff. DOI logo
Sun, Sanjun & Kairong Xiao
2019. Chinese scholarship in Cognitive Translation Studies. Translation, Cognition & Behavior 2:1 ► pp. 125 ff. DOI logo

This list is based on CrossRef data as of 7 november 2024. Please note that it may not be complete. Sources presented here have been supplied by the respective publishers. Any errors therein should be reported to them.