Cover not available
Part of
Reassessing Dubbing: Historical approaches and current trends
Edited by Irene Ranzato and Serenella Zanotti
[Benjamins Translation Library 148] 2019
► pp. 125–144

Chapter 6
Are we all together across languages?

An eye tracking study of original and dubbed films

In one of the first attempts to apply eye-tracking technology to the area of dubbing, this chapter reports on an experiment conducted in 2016 with viewers of the original English and dubbed Italian version of Wes Anderson’s The Grand Budapest Hotel. The study analysed the gaze behaviour of participants watching different scenes, focusing on variations in attentional synchrony and visual momentum. The results show less attentional synchrony and a higher visual momentum for the dubbing viewers. They also support the existence of the so-called dubbing effect, an unconscious eye movement strategy performed by dubbing viewers to avoid looking at mouths in dubbing, which prevails over the natural and idiosyncratic way in which they watch original films and real-life scenes.

Keywords: attentional synchrony, dubbing, dubbing effect, eye tracking, perception, reception, visual momentum
Article outline
  • 1.Introduction
  • 2.Key concepts in eye-tracking research and beyond
  • 3.Eye tracking and audiovisual translation
  • 4. The Grand Budapest Hotel: An eye-tracking study
    • 4.1Research questions and material
    • 4.2Experiment set up
    • 4.3Close ups and medium close ups; the dubbing effect
    • 4.4Written text in dubbed films
    • 4.5Action scenes and long shots
  • 5.Conclusion
  • References
  • Filmography
Published online: 6 August 2019
DOI logo
https://doi.org/10.1075/btl.148.06di
References
Alton, John
1949 Painting with Light. California: University of California Press.Google Scholar
Berger, John
1972 Ways of Seeing. London: Penguin Books.Google Scholar
Bisson, Marie-Josée, Walter Van Heuven, Kathy Conklin, and Richard Tunney
2014 “Processing of Native and Foreign Language Subtitles in Films: An Eye Tracking Study.” Applied Psycholinguistics 35 (2): 399–418. DOI logoGoogle Scholar
Chion, Michael
1994 Audio-Vision: Sound on Screen. New York: Columbia University Press.Google Scholar
Coleridge, Samuel T.
1871 Biographia Literaria, ed. by J. Shawcross, Oxford Clarendon Press.Google Scholar
Coutrot, Antoine, Nathalie Guyader, Gelu Ionescu, and Alice Caplier
2012 “Influence of Soundtrack on Eye Movements During Video Exploration”. Journal of Eye Movement Research 5(5): 1–10.Google Scholar
Crouzet, Sebastien M., Holle Kirchner, and Simon J. Thorpe
2010 “Fast Saccades Toward Faces: Face Detection in Just 100 ms”. Journal of Vision, 10 (4): 1–17. DOI logoGoogle Scholar
De Linde, Zoé, and Kay, Neil
1999 The Semiotics of Subtitling. Manchester, UK: St. Jerome Publishing.Google Scholar
d’Ydewalle, Géry, Johan Van Rensbergen, and Joris Pollet
1987 “Reading a Message when the Same Message Is Available Auditorily in Another Language: The Case of Subtitling”. In Eye Movements: From Physiology to Cognition, ed. by J.K O’Reagan and A. Lévy Schoen, 313–321. Amsterdam: Elsevier Science Publishers B.V. (North-Holland). DOI logoGoogle Scholar
d’Ydewalle, Géry, and Ingrid Gielen
1992 “Attention Allocation with Overlapping Sound, Image, and Text.” In Eye Movements and Visual Cognition: Scene Perception and Reading, ed. by Keith Rayner, 415–427. New York: Springer-Verlag. DOI logoGoogle Scholar
d’Ydewalle, Géry, and Wim De Bruycker
2007 “Eye Movements of Children and Adults while Reading Television Subtitles”. European Psychologist 12: 196–205. DOI logoGoogle Scholar
Dyer, Adrian G., and Sarah Pink
2015 “Movement, Attention and Movies: The Possibilities and Limitations of Eye Tracking?”. Refractory 25: online at [URL].Google Scholar
Dwyer, Tessa, Claire Perkins, Sean Redmond, and Jodi Sita
(eds.) 2018 Seeing into Screens: Eye Tracking and the Moving Image, Bloomsbury.Google Scholar
Foulsham, Tom, and Lucy Anne Sanderson
2013 “Look Who’s Talking? Sound Changes Gaze Behaviour in a Dynamic Social Scene. ” Visual Cognition 21 (7): 922–944. DOI logoGoogle Scholar
Hochberg, Jullian
1962 “The Psychophysics of Pictorial Perception.” Audio-Visual Communications Review 10: 22–54.Google Scholar
Hochberg, Julian, and Virginia Brooks
1978 “Film Cutting and Visual Momentum.” In Eye Movements and the Higher Psychological Functions, ed. by John W. Senders, Dennis F. Fisher, and Richard A. Monty, 293–313. Hillsdale: Laurence Erlbaum.Google Scholar
Hvelplund, Kristian Tangsgaard
2017 “Eye tracking and the process of dubbing translation.” In Fast-forwarding with Audiovisual Translation, ed. by Jorge Díaz Cintas, and Kristijan Nikolić, 110–125. London: Multilingual Matters.Google Scholar
Jensema, Carl, J. Sameh el Sharkawy, Ramalinga S. Danturthi, Robert Burch, and David Hsu
2000 “Eye Movement Patterns of Captioned Television Viewers”. American Annals of the Deaf 145 (3): 275–285. DOI logoGoogle Scholar
Land, Michael F., and David N. Lee
1994 “Where We Look when We Steer.” Nature 369: 742–744. DOI logoGoogle Scholar
Mercado, Gustavo
2010 The Filmmaker’s Eye: Learning (and Breaking) the Rules of Cinematic Composition. Amsterdam: Focal Press.Google Scholar
Perego, Elisa, Fabio del Missier, Marco Porta, and Mauro Mosconi
2010 “The Cognitive Effectiveness of Subtitle Processing”. Media Psychology 13: 243–272. DOI logoGoogle Scholar
Perego, Elisa, David Orrego-Carmona and Sara Bottiroli
2016 “An Empirical Take on the Dubbing vs Subtitling Debate. An Eye Movement Study”. In Lingue e Linguaggi 19: 255–274.Google Scholar
Rassell, Andrea, Redmond, Sean
, et al. 2016 “Seeing, Sensing Sound: Eye Tracking Soundscapes in Saving Private Ryan and Monster Inc.” In Making Sense of Cinema. Empirical Studies into Film Spectators and Spectatorship, ed. by CarrieLynn D. Reinhard, and Christopher J. Olson, 139–164. New York/London: Bloomsbury.Google Scholar
Redmond, Sean, and Craig Batty
2015 “Seeing into Things: Eye Tracking the Moving Image.” Refractory 25: online at [URL].Google Scholar
Robinson, Jennifer, Jane Stadler and Andrea Rassell
2015 “Sound and Sight: An Exploratory Look at Saving Private Ryan through the Eye Tracking Lens.” Refractory. A Journal of Entertainment Media 25. Available online: [URL] (accessed 1 October 2016).Google Scholar
(2015) “Sound and Sight: An Exploratory Look at Saving Private Ryan through the Eye-tracking Lens”. Refractory: a Journal of Entertainment Media 25. [URL] (consulted 20.09.2018).Google Scholar
Romero-Fresco, Pablo
2009 “Naturalness in the Spanish Dubbing Language: a Case of Not-so-close Friends.”. Meta 54 (1): 49–72.Google Scholar
Romero-Fresco, P.
forthcoming) “The dubbing effect: An eye-tracking study on how viewers make dubbing work”. Journal of Audiovisual Translation, Jostrans, 33, Special Issue: Experimental Research and Cognition in AVT.
Smith, Tim J., and John M. Henderson
2008 “Edit Blindness: The Relationship Between Attention and Global Change Blindness in Dynamic Scenes.” Journal of Eye Movement Research 2(2): 1–17.Google Scholar
Smith, Tim J.
2011 “Watching You Watch THERE WILL BE BLOOD.” Observations on Film Art. ed. by Kristin Thompson and David Bordwell. Online at: [URL].Google Scholar
2015 “Read, Watch, Listen. A Commentary on Eye Tracking and Moving Images.” Refractory 25: online at [URL].Google Scholar
Treuting, Jennifer
2006 “Eye Tracking and Cinema: A Study of Film Theory and Visual Perception”. Society of Motion Picture and Television Engineers 115 (1): 31–40.Google Scholar
Vilarò, Anna, and Tim J. Smith
(2011) “Subtitle Reading Effects on Visual and Verbal Information Processing in Films.” Published abstract In Perception. ECVP abstract supplement. 40:153. European Conference on Visual Perception. Toulouse, France.Google Scholar

Filmography

Casablanca
. Michael Curtiz. USA 1942.Google Scholar
The Grand Budapest Hotel
. Wes Anderson. UK/Germany 2014.Google Scholar
Cited by

Cited by 5 other publications

Brannon, William, Yogesh Virkar & Brian Thompson
2023. Dubbing in Practice: A Large Scale Study of Human Localization With Insights for Automatic Dubbing. Transactions of the Association for Computational Linguistics 11 ► pp. 419 ff. DOI logo
Di Giovanni, Elena
2020. Reception Studies and Audiovisual Translation. In The Palgrave Handbook of Audiovisual Translation and Media Accessibility [Palgrave Studies in Translating and Interpreting, ], ► pp. 397 ff. DOI logo
Hayes, Lydia
2023. English Dubs: Why are Anglophone Viewers Receptive to English Dubbing on Streaming Platforms and to Foreign-Accent Strategies?. Íkala, Revista de Lenguaje y Cultura 28:2 ► pp. 1 ff. DOI logo
Raffi, Francesca
2020. The Impact of Italian Dubbing on Viewers’ Immersive Experience: An Audience Reception Study. Online Journal of Communication and Media Technologies 10:3 ► pp. e202019 ff. DOI logo
Sánchez-Mompeán, Sofía
2023. Engaging English Audiences in the Dubbing Experience: A Matter of Quality or Habituation?. Íkala, Revista de Lenguaje y Cultura 28:2 ► pp. 1 ff. DOI logo

This list is based on CrossRef data as of 17 september 2023. Please note that it may not be complete. Sources presented here have been supplied by the respective publishers. Any errors therein should be reported to them.