Cover not available
Article published In:
Cognitive Translation and Interpreting Studies in the Early Twenty First Century
Edited by Adolfo M. García, Edinson Muñoz and Néstor Singer
[Translation, Cognition & Behavior 6:2] 2023
► pp. 211–229

Experimenting with audio description

When participants are all ears in CTIS

ORCID logo | University of Murcia
ORCID logo | University of Murcia

After years in the shadows of translation, research on audio description (AD) is now gaining momentum thanks to its contribution to a more accessible and inclusive world. This paper claims that experimental AD studies have a rightful place in cognitive translation and interpreting studies (CTIS) and discusses some of its limitations and prospects. The first section introduces the relevance and interest of choosing accessibility as a general field of study and AD as a modality of accessible translation. The second section takes a snapshot of existing experimental AD studies and summarizes their main results. Then, useful tips are provided to help confront relevant challenges. The final section discusses the potential of AD research for advancing our knowledge of sensory, multimodal, and cognitive processing and for transferring research results to society.

Keywords: audio description, cognitive translation and interpreting studies (CTIS), accessible translation
Article outline
  • 1.Meeting at the start line: The right to universal accessibility and accessible translation
  • 2.When the minority is heard: CTIS on AD
    • 2.1A quick glance at the elaboration process
    • 2.2Under the sharp eye of AV consumers
    • 2.3Within the attentive ear of AD consumers
  • 3.Confronting the weaknesses: Tips for experimental AD research in CTIS
  • 4.Closer to the finish line: Embracing the potential of AD research
  • Note
  • References
Published online: 7 March 2024
DOI logo
https://doi.org/10.1075/tcb.00086.roj
References
Bardini, Floriane
2013 Audio Description of Cinematic Language: A Comparative Study of Describers’ Approaches. Paper presented at the Advanced Research Seminar on Audio Description (ARSAD) on 13 – 14 March 2013.
2017 “Audio Description Style and the Film Experience of Blind Spectators: Design of a Reception Study”. Rivista Internazionale di Tecnica della Traduzione / International Journal of Translation 191: 49–73. DOI logo ISSN 1722–5906 (print) ISSN 2421–6763 (online).Google Scholar
2020 “Film language, film experience and film interpretation in a reception study comparing conventional and interpretative audio description styles”. In Innovation in Audio Description Research, edited by Sabine Sabine, and Kim Starr, 76–96). London: Routledge.Google Scholar
Barnés-Castaño, Celia, Luisa Bernstorff, and Carmen Vilches
2021 “Action Research in Motion for Dance Audio Description”. New Voices in Translation Studies 21: 1–26.Google Scholar
Barrouillet, Pierre, Sophie Bernardin, Sophie Portrat, Evie Vergauwe, and Valèrie Camos
2007 “Time and cognitive load in working memory”. Journal of Experimental Psychology: Learning, Memory and Cognition 33 (3): 570–85. DOI logo.Google Scholar
Barry, Robert J.
2009 “Habituation of the orienting reflex and the development of preliminary process theory”. Neurobiology of learning and memory 92 (2): 235–242.Google Scholar
Barry, Robert J., and Carlie Lawrence
2009 “ERPs and the evoked cardiac response to auditory stimuli: Intensity and cognitive load effects”. Acta Neurobiologiae Experimentalis 691: 552–559.Google Scholar
Benedek, Mathias, and Christian Kaernbach
2010 “A continuous measure of phasic electrodermal activity”. Journal of Neuroscience Methods 190 (1): 80–91.Google Scholar
Bläsing, Bettina, and Esther Zimmermann
2021 “Dance Is More Than Meets the Eye—How Can Dance Performance Be Made Accessible for a Non-sighted Audience?” Frontiers in Psychology 121: 643848. DOI logoGoogle Scholar
Borg, Claudine
2018 “The phases of the translation process: are they always three?” Proceedings of the Junior College multi-disciplinary conference: research, practice and collaboration. Breaking Barriers: annual conference in Malta: 79–92. University of Malta: Junior College Publishing.Google Scholar
Boucsein, Wolfram
2012 Electrodermal activity. Berlin: Springer Science & Business Media.Google Scholar
Bradley, Margaret M., and Peter J. Lang
1994 “Measuring emotion: the self-assessment manikin and the semantic differential”. Journal of Behavior Therapy and Experimental Psychiatry 25 (1): 49–59. DOI logoGoogle Scholar
Braun, Sabine
2007 “Audio Description from a discourse perspective: a socially relevant framework for research and training”. Linguistica Antverpiensia 357–369.Google Scholar
2011 “Creating coherence in Audio Description”. Meta: Journal des Traducteurs/ Meta: Translator’s Journal 56 (3): 645–662.Google Scholar
Brunken, Roland, Jan L. Plass, and Detlev Leutner
2003 “Direct measurement of cognitive load in multimedia learning”. Educational Psychologist 38 (1): 53–61.Google Scholar
Buchwald, Mikołaj, Szymon Kupiński, Adam Bykowski Joanna Marcinkowska, Dawid Ratajczyk, and Marcin Jukiewicz
2019 “Electrodermal activity as a measure of cognitive load: a methodological approach”. Signal Processing: Algorithms, Architectures, Arrangements, and Applications: 175–179.Google Scholar
Cabeza-Cáceres, Cristóbal
2013 Audiodescripció i recepció: efecte de la velocitat de narració, l’entonació i l’explicitació en la comprensió fílmica [Audio description and reception: effect of narration speed, intonation and explicitness on filmic comprehension] (PhD). Universitat Autònoma de Barcelona, [URL]
Camos, Valérie, and Sophie Portrat
2015 “The impact of cognitive load on delayed recall”. Psychonomic Bulletin & Review 22 (4): 1029–1034. DOI logoGoogle Scholar
Cavallo, Amelia
2015 “Seeing the word, hearing the image: the artistic possibilities of audio description in theatrical performance”. Research in Drama Education: The Journal of Applied Theatre and Performance, 20 (1): 125–134. DOI logoGoogle Scholar
Chafe, Wallace
(Ed.) 1980 The Pear Stories: Cognitive, cultural and linguistic aspects of narrative production. Norwood, New Jersey: Ablex.Google Scholar
Di Giovanni, Elena
2014 “Audio introduction meets audio description: an Italian experiment”. Intralinea 161: 1–15.Google Scholar
Dieleman, Gwendolyn C., Anja Huizink, Joke Tulen, Elisabeth Utens, Hanneke Creemers, Jan van der Ende, and Frank Verhulst
2015 “Alterations in HPA-axis and autonomic nervous system functioning in childhood anxiety disorders point to a chronic stress hypothesis”. Psychoneuroendocrinology 511: 135–150.Google Scholar
Directive (EU) 2019/882 of the European Parliament and of the Council of 17 April 2019 on the accessibility requirements for products and services)
[URL]
European Disability Strategy 2010–2020: A Renewed Commitment to a Barrier-Free Europe for Disabled Persons
[URL]
Ferziger, Naomi, Yossi Freier Dror, Lirit Gruber, Sara Nahari, Nofar Goren, Nurit Neustadt-Noy, Noomi Katz, and Asnat Bar-Haim Erez
2020 “Audio description in the theater: assessment of satisfaction and quality of the experience among individuals with visual impairment”. British Journal of Visual Impairment 381: 299–311. DOI logoGoogle Scholar
Fresno Cañada, Nazaret
2014 “Is a Picture Worth a Thousand Words? The Role of Memory in Audio Description”. Across Languages and Cultures 15 (1): 111–129.Google Scholar
Fryer, Louise
2013 Putting it into words: The impact of visual impairment on perception, experience and presence. PhD, London University [available at [URL]].
Fryer, Louise, and Freeman, Jonathan
2012 “Cinematic language and the description of film: Keeping AD users in the frame”. Perspectives: Studies in Translatology 211: 412–426.Google Scholar
Fryer, Louise, and Jonathan Freeman
2013 “Visual impairment and presence”. In Proceedings of the 2013 Inputs-Outputs Conference on An Interdisciplinary Conference on Engagement in HCI and Performance, edited by Harry Witchel, 1–5. New York City: ACM Press. DOI logoGoogle Scholar
2014 “Can you feel what I’m saying? The impact of verbal information on emotion elicitation and presence in people with a visual impairment”. In Proceedings of the International Society for Presence Research, edited by Anna Felnhofer, and Oswald Kothgassner, 99–107. Wien: Facultas.Google Scholar
Fryer, Louise, and Pablo Romero-Fresco
2014 “Audiointroductions”. In Audio Description: New perspectives illustrated, edited by Agnieszka Maszerowska, Anna Matamala, and Pilar Orero, 11–28. Amsterdam: John Benjamins.DOI logoGoogle Scholar
Holsanova, Jana
2019 “Uncovering scientific and multimodal literacy through audio description”. Journal of Visual Literacy 39 (3–4): 132–148.Google Scholar
Hughes, Ashley M., Gabriella Hancock, Shannon L. Marlow, Kimberly Stowers, and Eduardo Salas
2019 “Cardiac measures of cognitive workload: a meta–analysis”. Human factors 61 (3): 393–414.Google Scholar
Igareda, Paula, and Anna Matamala
2012 “Variations on the Pear Tree Experiment: different variables, new results?”. Perspectives. Studies in Translatology 20 (1): 103–123.Google Scholar
Iturregui-Gallardo, Gonzalo
2019 “Audio subtitling: Voicing strategies and their effect on emotional activation”. PhD, Universitat Autònoma de Barcelona [available at [URL]].
Iturregui-Gallardo, Gonzalo, and Jorge Luis Méndez-Ulrich
2019 “Towards the Creation of a Tactile Version of the Self-Assessment Manikin (T-SAM) for the Emotional Assessment of Visually Impaired People”. International Journal of Disability, Development and Education 671: 657–674. DOI logoGoogle Scholar
Iturregui-Gallardo, Gonzalo, and Anna Matamala
2021 “Audio subtitling: dubbing and voice-over effects and their impact on user experience”. Perspectives 29 (1): 64–83. DOI logoGoogle Scholar
Jankowska, Anna
2013 “Taking a British idea to Poland: Audio introductions for voiced-over films”. Paper presented at the 4th Advanced Research Seminar on Audio Description (ARSAD), Barcelona, Spain, 13th of March.
2021 “Audio describing films: A first look into the description process”. JosTrans: The Journal of Specialized Translation 36a1: 26–52.Google Scholar
Jankowska, Anna, Joanna Pilarczyk, Kinga Wołoszyn, and Michał Kunieckib
2022 “Enough is enough: how much intonation is needed in the vocal delivery of audio description?” Perspectives. DOI logoGoogle Scholar
Jiménez-Hurtado, Catalina, and Silvia Soler Gallego
2015 “Museum accessibility through translation: A corpus study of pictorial audio description”. In Audiovisual Translation: Taking Stock, edited by Jorge Díaz Cintas, and Josélia Neves (Eds.), 277–299. Cambridge Scholars Publishing.Google Scholar
Kruger, Jan-Louis
2012 “Making meaning in AVT: Eye tracking and viewer construction of narrative”. Perspectives: Studies in Translatology 20 (1): 67–86.Google Scholar
Larmuseau, Charlotte, Pieter Vanneste, Jan Cornelis, Piet Desmet, and Fien Depaepe
2019 “Combining physiological data and subjective measurements to investigate cognitive load during complex learning”. Frontline Learning Research 7 (2): 57–74.Google Scholar
Lindquist, Kristen A., and Lisa Barrett
2008a “Constructing emotion: The experience of fear as a conceptual act”. Psychological Science 191: 898–903. DOI logoGoogle Scholar
2008b “Emotional complexity”. In Handbook of emotions, edited by Lisa Feldman Barrett, Michael Lewis, and Jeannette M. Haviland-Jones, 513–530. The Guilford Press.Google Scholar
Loaiza, Vanessa, David McCabe, Jessie Youngblood, Nathan Rose, and Joel Myerson
2011 “The influence of levels of processing on recall from working memory and delayed recall tasks”. Journal of Experimental Psychology: Learning, Memory, and Cognition 37 (5): 1258.Google Scholar
López, Mariana Julieta, Gavin Cyril Kearney, and Krisztian Hofstadter
2018 “Audio Description in the UK: what works, what doesn’t and understanding the need for personalising access”. British Journal of Visual Impairment 36 (3): 1–18. ISSN 0264–6196.Google Scholar
Maczynska, Magdalena, and Agnieszka Szarkowska
2011 “Text-to-speech audio description with audio subtitling to a non-fiction film La Soufriere by Werner Herzog”. Paper presented at the conference Media4ALL 4 in London, organized by the Imperial College London on 28 June – 1 July 2011.
Margolies, Eleanor
2015 “Going to hear a dance: On audio describing”. Performance Research 20 (6): 17–23. DOI logoGoogle Scholar
Matamala, Anna, Anna Fernández, and Carla Ortiz-Boix
2013 “Enhancing sensorial and linguistic accessibility: further developments in the TECNACC and ALST projects”. Paper presented at the conference M4ALL5, 25–27th September 2013, Dubrovnik, Croatia.
Mazur, Iwona
2017 “Audio description crisis points: The idea of common European audio description guidelines revisited.” In Fast-Forwarding with Audiovisual Translation, edited by Jorge Díaz Cintas, and Kristijan Nikoliĉ, 127–140. Bristol: Multilingual Matters.Google Scholar
2014 “Gestures and facial expressions in audio description”. In Audio Description. New perspectives illustrated, edited by Anna Maszerowska, Anna Matamala, and Pilar Orero, 179–198. Amsterdam: John Benjamins.DOI logoGoogle Scholar
Mazur, Iwona, and Agnieszka Chmiel
2016 “Should Audio Description Reflect the Way Sighted Viewers Look at Films? Combining Eye-Tracking and Reception Study Data”. In Researching Audio Description New Approaches, edited by Anna Matamala, and Pilar Orero Clavero, 97–121. London: Palgrave Macmillan.Google Scholar
Mazur, Iwona, and Jan-Louis Kruger
2012 “Pear stories and audio description: Language, perception and cognition across cultures”. Perspectives: Studies in Translatology 20 (1): 1–3. DOI logoGoogle Scholar
Muñoz Martín, Ricardo, and Celia Martín de León
2018 Fascinatin' Rhythm-and Pauses in Translators' Cognitive Processes. Hermes 571: 29–47. DOI logo.Google Scholar
Mutlu-Bayraktar, Duygu, Veysel Cosgun, and Tugba Altan
2019 “Cognitive load in multimedia learning environments: A systematic review”. Computers & Education 1411, 103618.Google Scholar
Orero, Pilar
2021 “Leitura de Filmes para a Produçao de audiodescriçoes: uma palavra vale mais que mil imagens? [Reading Films to Produce Audio Descriptions: is a word worth a thousand pictures?]”. In Diversas faces da tradução na contemporaneidade, edited by Erica Lima, and Viviane Veras, 183–207. Campinas SP: Pontes Editores.Google Scholar
Orero, Pilar, and Anna Vilaró
2012 “Eye tracking Analysis of Minor Details in Films for Audio Description”. Multidisciplinarity in Audiovisual Translation 41, 295–319. DOI logoGoogle Scholar
Paas, Fred
1992 “Training strategies for attaining transfer of problem-solving skills in statistics: A cognitive load approach”. Journal of Educational Psychology, 841, 429–434. DOI logoGoogle Scholar
Perego, Elena
2014 L’audiodescrizione filmica per i ciechi e gli ipovedenti [Film audio description for the blind and visually impaired]. Trieste: EUT Edizioni Università di Trieste.Google Scholar
Posadas Rodríguez, Gala
2010 “Audio description as a complex translation process: A protocol”. In New Insights into Audiovisual Translation and Media Accessibility: Media for All 21, edited by Jorge Díaz Cintas, Anna Matamala, and Joselia Neves, 195–211. Amsterdam/New York: Rodopi.Google Scholar
Potter, Robert, and Paul Bolls
2012 Psychophysiological measurement and meaning: Cognitive and emotional processing of media. London: Routledge.Google Scholar
Ramos Caro, Marina
2015 “The emotional experience of films: does audio description make a difference?”. The Translator 211: 68–94. DOI logoGoogle Scholar
2016 “Testing audio-narration: the emotional impact of language in AD”. Perspectives: Studies in Translatology 241: 606–634. DOI logoGoogle Scholar
Ramos Caro, Marina, and Ana María Rojo López
2014 “Feeling Audio Description: Exploring the Impact of AD on Emotional Response”. Translation Spaces 31: 133–150. DOI logoGoogle Scholar
2020 “Analysing the AD process: Creativity, expertise and quality”. The Journal of Specialized Translation 331, Special issue on experimental research and cognition in Audiovisual Translation: 212–232.Google Scholar
Ramos Caro, Marina, Laura Espín López, and Ana María Rojo López
2021 “The psychophysiological impact of audio described porn”. Onomázein, Special Issue VIII – Emotions in Translation and Interpreting, 104–127.Google Scholar
Richardson, Daniel, Nicole Griffin, Lara Zaki, Auburn Stephenson, Jiachen Yan, Thomas Curry, Richard Noble, John Hogan, Jeremy I. Skipper, and Joseph T. Devlin
2020 “Engagement in video and audio narratives: contrasting self-report and physiological measures”. Scientific Reports 101, 11298. DOI logoGoogle Scholar
Rojo López, Ana María, Marina Ramos Caro, and Laura Espín López
2021 “Audio Described vs. Audiovisual Porn: Cortisol, Heart Rate and Engagement in Visually Impaired vs. Sighted Participants”. Frontiers in Psychology 121: 661452. DOI logoGoogle Scholar
Romero-Fresco, Pablo
2019 Accessible Filmmaking. Integrating translation and accessibility into the filmmaking process. London: Routledge.Google Scholar
Romero-Fresco, Pablo, and Louise Fryer
2013 “Could Audio Described Films benefit from Audio Introductions? An Audience Response Study”. Journal of Visual Impairment and Blindness 107 (4): 287–285.Google Scholar
Snyder, Joel
2005 The Visual Made Verbal: A Comprehensive Training Manual and Guide to the History and Applications of Audio Description. Ward Cove: Dog Ear Publishing. ISBN 1457527227.Google Scholar
Szarkowska, Agnieszka
2011 “Text-to-speech audio description: towards wider availability of AD”. The Journal of Specialized Translation 151: 142–162.Google Scholar
Szarkowska, Agnieszka, and Anna Jankowska
2012 “Text-to-speech audio description of voiced-over films. A case study of audio described Volver in Polish”. In Emerging topics in translation: Audio description, edited by Elena Perego, 81–94. Trieste: EUT Edizioni Università di Trieste.Google Scholar
Szarkowska, Agnieszka, and Pilar Orero
2014 The importance of sound for audio description. In Maszerowska, Anna, Anna Matamala and Pilar Orero (Eds.), Audio description: New perspectives illustrated, 121–140. Amsterdam: John Benjamins.DOI logoGoogle Scholar
Udo, John Patrick, and Deborah. I. Fels
2009 “Suit the Action to the Word, the Word to the Action: An Unconventional Approach to Describing Shakespeare’s Hamlet”. Journal of Visual Impairment & Blindness 103 (3): 178–183.Google Scholar
Udo, John Patrick, and Deborah I. Fels
2010a “Universal design on stage: live audidescription for theatrical performances”. Perspectives, Studies in Translatology 181: 189–203. DOI logoGoogle Scholar
2010b “Enhancing the entertainment experience of blind and low-vision theatregoers through touch tours”. Disability & Society 251: 231–240. DOI logoGoogle Scholar
Van der Wel, Pauline, and Henk van Steenbergen
2018 “Pupil dilation as an index of effort in cognitive control tasks: A review”. Psychonomic Bulletin & Review 25 (6): 2005–2015.Google Scholar
Vandaele, Jeroen
2012 “What Meets the Eye. Cognitive Narratology for Audio Description”. Perspectives 20 (1): 87–102. DOI logoGoogle Scholar
Vilaró, Anna, Andrew Duchowski, Pilar Orero, Tom Grindinger, Stephen Tetreault, and Elena di Giovanni
2012 “How sound is the Pear Tree Story? Testing the effect of varying audio stimuli on visual attention distribution”. Perspectives: Studies in Translatology 20 (1): 55–65.Google Scholar
Vilaró, Anna, and Pilar Orero
2013 “User-centric cognitive assessment. Evaluation of attention in special working centres: from paper to Kinect”. Advances in Distributed Computing and Artificial intelligence Journal 2 (7): 19–22.Google Scholar
Walczak, Agnieszka
2017 Immersion in Audio Description: The Impact of Style and Vocal Delivery on Users’ Experience. Unpublished PhD Thesis. [URL]].
Walczak, Agnieszka, and Agnieszka Szarkowska
2012 “Text-to-speech audio description to educational materials for visually-impaired children”. In Audiovisual Translation across Europe, edited by Silvia Bruti, and Elena di Giovanni, 209–233. Bern: Peter Lang.Google Scholar
Walczak, Agnieszka, and Louise Fryer
2017 “Creative description: The impact of audio description style on presence in visually impaired audiences”. British Journal of Visual Impairment 351: 6–17. DOI logoGoogle Scholar
2018 “Vocal delivery of audio description by genre: measuring users’ presence”. Perspectives 26 (1): 69–83. DOI logoGoogle Scholar
Westgate, Erin C., and Timothy D. Wilson
2018 “Boring thoughts and bored minds: The MAC model of boredom and cognitive engagement”. Psychological Review, 125 (5): 1–25.Google Scholar
Wilken, Nicole, and Jan-Louis Kruger
2016 “Putting the audience in the picture: Mise-en-shot and psychological immersion in audio described film”. Across Languages and Cultures 17 (2): 251–270. DOI logoGoogle Scholar