Cover not available
Article published In:
Methodological Issues in Experimental Research in Audiovisual Translation and Media Accessibility
Edited by Gian Maria Greco, Anna Jankowska and Agnieszka Szarkowska
[Translation Spaces 11:1] 2022
► pp. 60–88

Using linear mixed models to analyze data from eye-tracking research on subtitling

ORCID logo | University of Warsaw
| Aston University | University of the Free State
| University of Warsaw
In this paper, we aim to promote the use of linear mixed models (LMMs) in eye-tracking research on subtitling. Using eye tracking to study viewers’ reading of subtitles often warrants controlling for many confounding variables. However, even assuming that these variables are known to researchers, such control may not be possible or desired. Traditional statistical methods such as t-tests or ANOVAs exacerbate the problem due to the use of aggregated data: each participant has one data point per dependent variable. As a solution, we propose the use of LMMs, which are better suited to account for a number of subtitle and participant characteristics, thus explaining more variance. We introduce essential theoretical aspects of LMMs and highlight some of their advantages over traditional statistical methods. To illustrate our point, we compare two analyses of the same dataset: one using a t-test; another using LMMs.
Keywords: eye tracking, subtitling, linear mixed-effects models, LMMs, traditional statistical methods, confounding variables, reading, audiovisual translation
Article outline
  • 1.Introduction
  • 2.Variables to control for in eye-tracking research on subtitling
  • 3.Problems with more common approaches to statistical analyses
  • 4.An introduction to linear mixed models
    • 4.1Fixed and random effects
    • 4.2Random intercepts and slopes
    • 4.3Fitting linear mixed models
  • 5.An example quasi-experiment: Comparing t-tests to LMMs
    • 5.1Research design
  • 6.Results with a more traditional approach: Paired-samples t-tests
  • 7.Analyzing the data with linear mixed models
    • 7.1Fitting the model
    • 7.2Variables included in the model and data analysis
    • 7.3Results
  • 8.Conclusions, limitations, and research recommendations
  • Notes
  • References
Published online: 14 June 2022
DOI logo
https://doi.org/10.1075/ts.21013.sil
References (73)
References
Baayen, Harald. 2008a. Analyzing Linguistic Data: A Practical Introduction to Statistics Using R. Cambridge: Cambridge University Press. DOI logoGoogle Scholar
. 2008b. Exploratory Data Analysis: An Introduction to R for the Language Sciences. Cambridge: Cambridge University Press.Google Scholar
Balling, Laura Winther. 2008. “A Brief Introduction to Regression Designs and Mixed-Effects Modelling by a Recent Convert.” Copenhagen Studies in Language, 175–92.Google Scholar
Balling, Laura Winther, and Kristian Tangsgaard Hvelplund. 2015. “Design and Statistics in Quantitative Translation (Process) Research”. Translation Spaces 4 (1): 170–187. DOI logoGoogle Scholar
Barr, Dale. J., Roger Levy, Christoph Scheepers, and Harry J. Tily. 2013. “Random Effects Structure for Confirmatory Hypothesis Testing: Keep It Maximal.” J Mem Lang 68 (3), 255–278. DOI logoGoogle Scholar
Bates, Douglas, Reinhold Kliegl, Shravan Vasishth, and R. Harald Baayen. 2018. “Parsimonius Mixed Models.” arXiv:1506.04967v2 [stat.ME].Google Scholar
Bisson, Marie-Josée, Walter J. B. Heuven, Kathy Conklin, and Richard J. Tunney. 2014. “The Role of Repeated Exposure to Multimodal Input in Incidental Acquisition of Foreign Language Vocabulary.” Language Learning 64 (4): 855–77. DOI logoGoogle Scholar
Bolker, Ben. 2021. “GLMM FAQ.” 2021. [URL]
Brown, Violet A. 2021. An Introduction to Linear Mixed-Effects Modelling in R. Advances in Methods and Practices in Psychological Science 4 (1): 1–19. DOI logoGoogle Scholar
Brysbaert, Marc, and Boris New. 2009. “Moving beyond Kučera and Francis: A Critical Evaluation of Current Word Frequency Norms and the Introduction of a New and Improved Word Frequency Measure for American English.” Behavior Research Methods 41 (4): 977–90. DOI logoGoogle Scholar
Burnham, Denis, Kaoru Sekiyama, Gerard Bailly, Pascal Perrier, and Eric Vatikiotis-Bateson. 2012. “Investigating Auditory-Visual Speech Perception Development.” In Audiovisual Speech Processing, edited by Gérard Bailly, Pascal Perrier and Eric Vatikiotis-Bateson, 62–75. Cambridge: Cambridge University Press. DOI logoGoogle Scholar
Cambra, Cristina, Olivier Penacchio, Núria Silvestre, and Aurora Leal. 2014. “Visual Attention to Subtitles When Viewing a Cartoon by Deaf and Hearing Children: An Eye-Tracking Pilot Study.” Perspectives 22 (4): 607–17. DOI logoGoogle Scholar
Carson, Robyn J., and Christina M. L. Beeson. 2013. “Crossing Language Barriers: Using Crossed Random Effects Modelling in Psycholinguistics Research.” Tutorials in Quantitative Methods for Psychology 9 (1): 25–41. DOI logoGoogle Scholar
Crossley, Scott A., Tom Cobb, and Danielle S. McNamara. 2013. “Comparing Count-Based and Band-Based Indices of Word Frequency: Implications for Active Vocabulary Research and Pedagogical Applications.” System 41 (4): 965–81. DOI logoGoogle Scholar
Cunnings, Ian, and Ian Finlayson. 2015. “Mixed Effects Modeling and Longitudinal Data Analysis.” In Advancing Quantitative Methods in Second Language Research, edited by Luke Plonsky, 159–81. New York: Routledge. DOI logoGoogle Scholar
De Bruycker, Wim, and Gery d’Ydewalle. 2003. “Reading Native and Foreign Language Television Subtitles in Children and Adults.” In The Mind’s Eye: Cognitive and Applied Aspects of Eye Movement Research, edited by Jukka Hyönä, Ralph Radach, and Heiner Deubel, 671–84. Amsterdam: North-Holland. DOI logoGoogle Scholar
Dijkstra, Ton, Koji Miwa, Bianca Brummelhuis, Maya Sappelli, and Harald Baayen. 2010. “How Cross-Language Similarity and Task Demands Affect Cognate Recognition.” Journal of Memory and Language 62 (3): 284–301. DOI logoGoogle Scholar
Doherty, Stephen, and Jan-Louis Kruger. 2018. “The Development of Eye Tracking in Empirical Research on Subtitling and Captioning.” In Seeing into Screens: Eye Tracking and the Moving Imaging, edited by Jodi Sita, Tessa Dwyer, Sean Redmond, and Claire Perkins, 46–64. London: Bloomsbury. DOI logoGoogle Scholar
Duyck, Wouter, Eva Van Assche, Denis Drieghe, and Robert J. Hartsuiker. 2007. “Visual Word Recognition by Bilinguals in a Sentence Context: Evidence for Nonselective Lexical Access.” J Exp Psychol Learn Mem Cogn 33 (4): 663–79. DOI logoGoogle Scholar
Díaz-Cintas, Jorge and Aline Remael (2021). Subtitling. Concepts and Practices. London and New York, Routledge.Google Scholar
Ellis, Nick C., and Alan Beaton. 1993. “Psycholinguistic Determinants of Foreign Language Vocabulary Learning.” Language Learning 43 (4): 559–617. DOI logoGoogle Scholar
Field, Andy. 2017. Discovering Statistics Using IBM SPSS Statistics. 5th ed. London: SAGE Publications, Inc.Google Scholar
Garson, G. David. 2020. Multilevel modelling: Applications in Stata®, IBM® SPSS®, SAS®, R, & HLMTM . New York: Sage.Google Scholar
Gerber-Morón, Olivia, and Agnieszka Szarkowska. 2018. “Line Breaks in Subtitling: An Eye Tracking Study on Viewer Preferences.” Journal of Eye Movement Research 11 (3): 1–22. DOI logoGoogle Scholar
Gerber-Morón, Olivia, Agnieszka Szarkowska, and Bencie Woll. 2018. “The Impact of Text Segmentation on Subtitle Reading.” Journal of Eye Movement Research 11 (4): 1–18. DOI logoGoogle Scholar
Ghia, Elisa. 2012. “The Impact of Translation Strategies on Subtitle Reading.” In Eye Tracking in Audiovisual Translation, edited by Elisa Perego, 155–82. Roma: Aracne Editrice.Google Scholar
Gile, Daniel. 2016. “Experimental research.” In Researching Translation and Interpreting, edited by Claudia V. Angelelli and Brian James Baer, 220–228. New York: Routledge.Google Scholar
Hajduk, Gabriela K. 2019. “ Introduction to Linear Mixed Models .” [URL]
Harrison, Xavier. A., Lynda Donaldson, Maria E. Correa-Cano, Julian Evans, David N. Fisher, Cecily E. D. Goodwin, Beth. S. Robinson, David J. Hodgson, and Richard Inger. 2018. “A Brief Introduction to Mixed Effects Modelling and Multi-Model Inference in Ecology.” PeerJ 61: e4794. DOI logoGoogle Scholar
Heck, Ronald H., Scott L. Thomas, and Lynn N. Tabata. 2012. Multilevel Modelling of Categorical Outcomes Using IBM SPSS. Quantitative Methodology Series. New York / London: Routledge.Google Scholar
Hedeker, Donald. 2003. “A Mixed-Effects Multinomial Logistic Regression Model.” Statistics in Medicine 22 (9): 1433–46. DOI logoGoogle Scholar
Holmqvist, Kenneth, Marcus Nyström, Richard Andersson, Richard Dewhurst, Halszka Jarodzka, and Joost van de Weijer. 2011. Eye Tracking: A Comprehensive Guide to Methods and Measures. Oxford: Oxford University Press.Google Scholar
Inhoff, Albrecht W., and Keith Rayner. 1986. “Parafoveal Word Processing during Eye Fixations in Reading: Effects of Word Frequency.” Perception & Psychophysics 40 (6): 431–39. DOI logoGoogle Scholar
Jensema, Carl. 1998. “Viewer Reaction to Different Television Captioning Speeds.” American Annals of the Deaf 143 (4): 318–24. DOI logoGoogle Scholar
Koolstra, Cees M., Tom H. A. Van Der Voort, and Gery d’Ydewalle. 1999. “Lengthening the Presentation Time of Subtitles on Television: Effects on Children’s Reading Time and Recognition.” Communications 24 (4): 407–22. DOI logoGoogle Scholar
Krejtz, Izabela, Agnieszka Szarkowska, and Maria Łogińska. 2016. “Reading Function and Content Words in Subtitled Videos.” Journal Of Deaf Studies And Deaf Education 21 (2): 222–32. DOI logoGoogle Scholar
Kruger, Jan-Louis. 2018. “Eye tracking in audiovisual translation research”. In The Routledge Handbook of Audiovisual Translation, edited by L. Pérez-González, 350–366. Routledge. DOI logoGoogle Scholar
Kruger, Jan-Louis, Esté Hefer, and Gordon Matthew. 2014. “Attention Distribution and Cognitive Load in a Subtitled Academic Lecture: L1 vs. L2.” Journal of Eye Movement Research 7 (5): 1–15. DOI logoGoogle Scholar
Kruger, Jan-Louis, Stephen Doherty, and María T. Soto-Sanfiel. 2017. “Original Language Subtitles: Their Effects on the Native and Foreign Viewer.” Comunicar, 501 (January): 23–32. DOI logoGoogle Scholar
Kruger, Jan-Louis, and Stephen Doherty. 2018. “Triangulation of Online and Offline Measures of Processing and Reception in AVT.” In Reception Studies and Audiovisual Translation, edited by Elena Di Giovanni and Yves Gambier, 91–109. Amsterdam/New York: John Benjamins. DOI logoGoogle Scholar
Lemhöfer, Kristin, and Ton Dijkstra. 2004. “Recognizing Cognates and Interlingual Homographs: Effects of Code Similarity in Language-Specific and Generalized Lexical Decision.” Memory & Cognition 32 (4): 533–50. DOI logoGoogle Scholar
Liao, Sixin, Jan-Louis Kruger, and Stephen Doherty. 2020. “The Impact of Monolingual and Bilingual Subtitles on Visual Attention, Cognitive Load, and Comprehension.” Journal of Specialised Translation 331: 70–98.Google Scholar
Liao, Sixin, Lili Yu, Erik D. Reichle, and Jan-Louis Kruger. 2020. “Using Eye Movements to Study the Reading of Subtitles in Video.” Scientific Studies of Reading 25 (5): 417–435. DOI logoGoogle Scholar
Łuczak, Krzysztof. 2017. “The Effects of the Language of the Soundtrack on Film Comprehension, Cognitive Load and Subtitle Reading Patterns. An Eye-Tracking Study.” Institite of Applied Linguistics. Warsaw: University of Warsaw.Google Scholar
Mellinger, Christopher D., and Thomas A. Hanson. (2017). Quantitative Research Methods in Translation and Interpreting Studies. New York: Routledge.Google Scholar
Meteyard, Lotte, and Robert A. I. Davies. 2020. “Best Practice Guidance for Linear Mixed-Effects Models in Psychological Science.” Journal of Memory and Language 1121. DOI logoGoogle Scholar
Moran, Siobhan. 2009. “The Effect of Linguistic Variation on Subtitle Reception.” Toronto: York University.Google Scholar
Muñoz, Carmen. 2017. “The Role of Age and Proficiency in Subtitle Reading. An Eye-Tracking Study.” System 671: 77–86. DOI logoGoogle Scholar
Orrego-Carmona, David. 2015. “The Reception of (Non)Professional Subtitling.” Department of English and German Studies. Tarragona: Universitat Rovira i Virgili.Google Scholar
Peters, Elke. 2019. “Factors Affecting the Learning of Single-Word Items.” In The Routledge Handbook of Vocabulary Studies, edited by Stuart Webb, 125–42. New York: Routledge. DOI logoGoogle Scholar
Ragni, Valentina. 2020. “More than Meets the Eye: An Eye-Tracking Study of the Effects of Translation on the Processing and Memorisation of Reversed Subtitles.” Journal of Specialised Translation 331: 99–128.Google Scholar
Rajendran, Dhevi J., Andrew T. Duchowski, Pilar Orero, Juan Martínez, and Pablo Romero-Fresco. 2013. “Effects of Text Chunking on Subtitling: A Quantitative and Qualitative Examination.” Perspectives 21 (1): 5–21. DOI logoGoogle Scholar
Rayner, Keith. 2009. “Eye movements and attention in reading, scene perception, and visual search”. The Quarterly Journal of Experimental Psychology 62(8); 1457–1506. DOI logoGoogle Scholar
Rayner, Keith, and Susan A. Duffy. 1986. “Lexical Complexity and Fixation Times in Reading: Effects of Word Frequency, Verb Complexity, and Lexical Ambiguity.” Memory & Cognition 14 (3): 191–201. DOI logoGoogle Scholar
Rayner, Keith, Tessa Warren, Barbara J. Juhasz, and Simon P. Liversedge. 2004. “The Effect of Plausibility on Eye Movements in Reading.” J Exp Psychol Learn Mem Cogn 30 (6): 1290–1301. DOI logoGoogle Scholar
Rayner, K., Timothy J. Slattery, Denis Drieghe, and Simon P. Liversedge. 2011. “Eye Movements and Word Skipping during Reading: Effects of Word Length and Predictability.” J Exp Psychol Hum Percept Perform 37 (2): 514–28. DOI logoGoogle Scholar
Romero-Fresco, Pablo. 2015. “Final Thoughts: Viewing Speed in Subtitling.” In The Reception of Subtitles for the Deaf and Hard of Hearing in Europe, edited by Pablo Romero-Fresco, 335–41. Bern: Peter Lang.Google Scholar
RStudio Team. 2021. Studio: Integrated Development for R. RStudio. R. Boston. [URL]
Saldanha, Gabriela, and Sharon O’Brien. 2014. Research Methodologies in Translation Studies. New York: Routledge. DOI logoGoogle Scholar
Schwartz, Ana I., and Judith F. Kroll. 2006. “Bilingual Lexical Activation in Sentence Context.” Journal of Memory and Language 55 (2): 197–212. DOI logoGoogle Scholar
Singmann, Henrik, and David Kellen. 2019. “An Introduction to Mixed Models for Experimental Psychology.” In New Methods in Cognitive Psychology, edited by Daniel Spieler and Eric Schumacher, 1st ed., 4–31. Routledge. DOI logoGoogle Scholar
Silva, Breno B., David Orrego-Carmona, and Agnieszka Szarkowska. 2021a. Data for paper entitled ‘Using linear mixed models to analyze data from eye-tracking research on subtitling’ Figshare . Available at: (accessed March 2022). DOI logoGoogle Scholar
. 2021b. Appendices for paper entitled ‘Using linear mixed models to analyze data from eye-tracking research on subtitling’ Figshare . Available at: (accessed March 2022). DOI logoGoogle Scholar
Szarkowska, Agnieszka, Izabela Krejtz, Zuzanna Kłyszejko, and Anna Wieczorek. 2011. “Verbatim, Standard, or Edited? Reading Patterns of Different Captioning Styles among Deaf, Hard of Hearing, and Hearing Viewers.” American Annals of the Deaf 156 (4): 363–78. DOI logoGoogle Scholar
Szarkowska, Agnieszka, and Olivia Gerber-Morón. 2018. “Viewers Can Keep up with Fast Subtitles: Evidence from Eye Movements.” Plos One 13 (6). DOI logoGoogle Scholar
. 2019. “Two or Three Lines: A Mixed-Methods Study on Subtitle Processing and Preferences.” Perspectives 27 (1): 144–64. DOI logoGoogle Scholar
Szarkowska, Agnieszka, and Lidia Bogucka. 2019. “Six-Second Rule Revisited: An Eye-Tracking Study on the Impact of Speech Rate and Language Proficiency on Subtitle Reading.” Translation, Cognition & Behavior 2 (1): 101–24. DOI logoGoogle Scholar
Szarkowska, Agnieszka, Breno B. Silva, and David Orrego-Carmona. 2021. Effects of subtitle speed on proportional reading time: Re-analysing subtitle reading data with mixed effects models. Translation, Cognition & Behavior, 4(2): 305–330. DOI logoGoogle Scholar
“UCLA Institute of Digital Research.” 2021. FAQ How Do I Interpret a Regression Model When Some Variables Are Log Transformed? 2021. [URL]
Winter, Bodo. 2020. Statistics for Linguists: An Introduction Using R. New York / London: Routledge.Google Scholar
Ydewalle, Géry d’, Johan van Rensbergen, and Joris Pollet. 1987. “Reading a Message When the Same Message Is Available Auditorily in Another Language: The Case of Subtitling.” In Eye Movements: From Physiology to Cognition, edited by J. K. O’Regan and A. Levy-Schoen, 313–21. Amsterdam/New York: Elsevier. DOI logoGoogle Scholar
Ydewalle, Géry d’, Caroline Praet, Karl Verfaillie, and Johan Van Rensbergen. 1991. “Watching Subtitled Television: Automatic Reading Behavior.” Communication Research 18 (5): 650–66. DOI logoGoogle Scholar
Ydewalle, Géry d’, and Wim De Bruycker. 2007. “Eye Movements of Children and Adults While Reading Television Subtitles.” European Psychologist 12 (3): 196–205. DOI logoGoogle Scholar
Cited by (7)

Cited by seven other publications

Balaskas, Stefanos, Maria Koutroumani, Maria Rigou & Spiros Sirmakessis
2024. From Eye Movements to Personality Traits: A Machine Learning Approach in Blood Donation Advertising. AI 5:2 ► pp. 635 ff. DOI logo
Berlin Khenis, A.A., A.N. Puchkova, E.S. Kashchenko & M.Yu. Lebedeva
2024. Metacognitive Reading Strategies: Analysis of Self-report Data and Oculomotor Behaviour of Russian Schoolchildren. Psychological Science and Education 29:2 ► pp. 15 ff. DOI logo
Das, Anwesha, Zekun Wu, Iza Skrjanec & Anna Maria Feit
2024. Shifting Focus with HCEye: Exploring the Dynamics of Visual Highlighting and Cognitive Load on User Attention and Saliency Prediction. Proceedings of the ACM on Human-Computer Interaction 8:ETRA ► pp. 1 ff. DOI logo
Hrabčáková, Adriána
2023. Some Current Trends and Research Possibilities in the Audiovisual Translation of the 2020s. Language Culture Politics International Journal 1:1/2023 ► pp. 49 ff. DOI logo
Hrabčáková, Adriána
2023. Some Current Trends and Research Possibilities in the Audiovisual Translation of the 2020s. Language Culture Politics International Journal 1:1/2023 ► pp. 49 ff. DOI logo
Greco, Gian Maria, Anna Jankowska & Agnieszka Szarkowska
2022. Addressing methodological issues in experimental research in audiovisual translation. Translation Spaces 11:1 ► pp. 1 ff. DOI logo
Silva, Breno B., David Orrego-Carmona & Agnieszka Szarkowska
2022. Using linear mixed models to analyze data from eye-tracking research on subtitling. Translation Spaces 11:1 ► pp. 60 ff. DOI logo

This list is based on CrossRef data as of 28 september 2024. Please note that it may not be complete. Sources presented here have been supplied by the respective publishers. Any errors therein should be reported to them.