Cover not available
Article published In:
ITL - International Journal of Applied Linguistics
Vol. 174:2 (2023) ► pp.263–290

Measuring the visual in audio-visual input

The effects of imagery in vocabulary learning through TV viewing

ORCID logo | University of Barcelona
ORCID logo | University of Barcelona

This exploratory study investigates the effects of imagery on word learning through audio-visual input. A total of 82 adolescent EFL learners were exposed to 8 episodes of a TV series under four conditions, depending on the language of the on-screen text (L1 or L2) and whether they were pre-taught target words or not. The effects of co-occurrence of the word with its image, and the image time on screen (ITOS) were explored, alongside frequency, proficiency, and learning condition variables. Results showed that both image-related variables and frequency predicted word-form learning, while only ITOS predicted word-meaning recall, with a longer exposure to image associated to higher gains, suggesting that, at this age and proficiency level, the images associated with the words can be conducive to learning.

Keywords: imagery, TV viewing, frequency, vocabulary learning, adolescents
Article outline
  • Background
    • TV series and vocabulary learning
    • TV series and imagery
    • Frequency of occurrence in viewing
  • Aim and research questions
  • Methodology
    • Participants
    • Audio-visual materials
    • Instruments
    • Imagery measures
    • Procedure
    • Scoring of vocabulary tests
    • Preliminary analysis
  • Results
    • Word-form learning
    • Word-meaning learning
  • Discussion
    • Imagery
    • Frequency of encounters
    • Learning conditions and learner variables
  • Conclusions
  • Notes
  • References
Published online: 6 March 2023
DOI logo
https://doi.org/10.1075/itl.22019.puj
References
Avello, D. & Muñoz, C.
forthcoming). The use of captioned-video viewing to support the development of L2 reading skills. Proceedings of the fifth ELTRIA Conference, Barcelona.
Baltova, I.
(1994) The impact of video on the comprehension skills of core French students. Canadian Modern Language Review, 50 (3), 507–31. DOI logoGoogle Scholar
Bianchi, F., & Ciabattoni, T.
(2008) Captions and subtitles in EFL learning: an investigative study in a comprehensive computer environment. In Baldry, A., Pavesi, M., & Taylor Torsello, C. (Eds.) From didactas to ecolingua: An ongoing research project on translation and corpus linguistics (pp. 69–90). Trieste: Edizioni Università di Trieste. [URL]
Boers, F.
(2018) Intentional versus incidental learning. The TESOL Encyclopedia of English Language Teaching, 1–6. DOI logoGoogle Scholar
Brown, R., Waring, R., & Donkaewbua, S.
(2008) Incidental vocabulary acquisition from reading, reading-while-listening, and listening to stories. Reading in a foreign language, 20 (2), 136–163.Google Scholar
Brysbaert, M., Warriner, A. B., & Kuperman, V.
(2014) Concreteness ratings for 40 thousand generally known English word lemmas. Behavior research methods, 46 (3), 904–911. DOI logoGoogle Scholar
Chai, J., & Erlam, R.
(2008) The effect and the influence of the use of video and captions on second language learning. New Zealand Studies in Applied Linguistics, 14 (2), 25.Google Scholar
Charles, T., & Trenkic, D.
(2015) Speech segmentation in a second language: The role of bimodal input. In Gambier, Y., Caimi, A., & Mariotti, C. (Eds.) Subtitles and language learning: Principles, strategies and practical experiences (pp. 173–198). Bern: Peter Lang.Google Scholar
Chen, Y. R., Liu, Y. T., & Todd, A. G.
(2018) Transient but effective? Captioning and adolescent EFL learners’ spoken vocabulary acquisition. English Teaching & Learning, 421, 25–56. DOI logoGoogle Scholar
Danan, M.
(2004) Captioning and subtitling: undervalued language learning strategies. Meta: Journal Des Traducteurs, 49 (1), 67–77. DOI logoGoogle Scholar
Deconinck, J., Boers, F., & Eyckmans, J.
(2017) ‘Does the form of this word fit its meaning?’ The effect of learner-generated mapping elaborations on L2 word recall. Language teaching research, 21 (1), 31–53. DOI logoGoogle Scholar
Douglas Fir Group
(2016) A transdisciplinary framework for SLA in a multilingual world. The Modern Language Journal, 100 (S1), 19–47. DOI logoGoogle Scholar
Durbahn, M., Rodgers, M., & Peters, E.
(2020) The relationship between vocabulary and viewing comprehension. System, 88 1. DOI logoGoogle Scholar
Elley, W. B.
(1989) Vocabulary acquisition from listening to stories. Reading Research Quarterly 24 (2), 174–187. [URL]. DOI logo
Feng, Y., & Webb, S.
(2020) Learning vocabulary through reading, listening, and viewing: Which mode of input is most effective? Studies in Second Language Acquisition, 42 (3), 499–523. DOI logoGoogle Scholar
Gullberg, M., De Bot, K., & Volterra, V.
(2008) Gestures and some key issues in the study of language development. Gestures 8 (2), 149–179. DOI logoGoogle Scholar
Hasan, A.
(2000) Learners’ perceptions of listening comprehension problems. Language, Culture and Curriculum, 13 (2), 137–53. DOI logoGoogle Scholar
Horst, M., Cobb, T., & Meara, P.
(1998) Beyond a clockwork orange: Acquiring second language vocabulary through reading. Reading in a Foreign Language 11 (2), 207–23.Google Scholar
Hulstijn, J. H.
(2003) Incidental and Intentional Learning. In Doughty, C. & Long, M. H. (Eds.), The handbook of second language acquisition (Vol. 191, pp. 349–381). MA: Blackwell. DOI logoGoogle Scholar
(2013) Incidental learning in second language acquisition. In Chapelle, C. A. (Ed.) The encyclopedia of applied linguistics (Vol. 51, pp. 2632–40). Chichester: Wiley-Blackwell. DOI logoGoogle Scholar
Jelani, N. A., & Boers, F.
(2018) Examining incidental vocabulary acquisition from captioned video: Does test modality matter? ITL-International Journal of Applied Linguistics, 169 (1), 169–190. DOI logoGoogle Scholar
Jones, L.
(2004) Testing L2 vocabulary recognition and recall using pictorial and written test items. Language learning & technology 8 (3), 122–143. ISSN 1094-3501Google Scholar
Jones, L., & Plass, J.
(2002) Supporting listening comprehension and vocabulary acquisition in French with multimedia annotations. The Modern Language Journal, 86 (4), 546–61. DOI logoGoogle Scholar
Khan, N., Kasdar, J., Melvin, M., Blomquist, R., Huang, E., & McEwen, J.
(2015) Fresh off the Boat [TV series] Los Angeles, CA: ABC.Google Scholar
Krashen, S. D.
(2003) Explorations in language acquisition and use. Portsmouth, NH: Heinemann. ISBN: 0-325-00554-0Google Scholar
Lee, S.
(2007) Effects of textual enhancement and topic familiarity on Korean EFL students’ reading comprehension and learning of passive form. Language Learning, 57 (1), 87–118. DOI logoGoogle Scholar
Lindgren, E., & Muñoz, C.
(2013) The influence of exposure, parents, and linguistic distance on young European learners’ foreign language comprehension. International Journal of Multilingualism, 10 (1), 105–129. DOI logoGoogle Scholar
Maleki, A., & Safaee Rad, M.
(2011) The effect of visual and textual accompaniments to verbal stimuli on the listening comprehension test performance of Iranian high and low proficient EFL learners. Theory and Practice in Language Studies, 1 (1), 28–36. DOI logoGoogle Scholar
Markham, P.
(1999) Captioned videotapes and second-language listening word recognition. Foreign Language Annals, 32 (3), 321–28. DOI logoGoogle Scholar
Mayer, R.
(2014) The Cambridge handbook of multimedia learning. Cambridge: Cambridge University Press. DOI logoGoogle Scholar
Meara, P., & Milton, J.
(2003) X_Lex: the Swansea levels test. Berkshire: Express Publishing.Google Scholar
Miralpeix, I.
(2012, March). X_Lex and Y_Lex: A validation study. Paper presented at the 22nd Lexical Studies Conference, Newtown, UK.
Montero Perez, M.
(2019) Pre-learning vocabulary before viewing captioned video: an eye-tracking study. The Language Learning Journal, 47 (4), 460–478. DOI logoGoogle Scholar
Montero Perez, M., Peters, E., & Desmet, P.
(2018) Vocabulary learning through viewing video: the effect of two enhancement techniques. Computer Assisted Language Learning, 31 (1–2), 1–26. DOI logoGoogle Scholar
Montero Perez, M., Peters, E., Clarebout, G., & Desmet, P.
(2014) Effects of captioning on video comprehension and incidental vocabulary learning. Language Learning and Technology, 18 (1), 118–41. DOI logoGoogle Scholar
Muñoz, C., Pujadas, G., & Pattemore, A.
(2023) Audio-visual input for learning L2 vocabulary and grammatical constructions. Second Language Research 39 ( 1 ), 13–38. DOI logoGoogle Scholar
Naghizadeh, M., & Darabi, T.
(2015) The impact of bimodal, Persian and no-subtitle movies on Iranian EFL learners’ L2 vocabulary learning. Journal of Applied Linguistics and Language Research, 2 (2), 66–79. DOI logoGoogle Scholar
Nation, P.
(2015) Principles guiding vocabulary learning through extensive reading. Reading in a Foreign Language 27 (1): 136–145.Google Scholar
Nation, P., & Heatley, A.
(2002) Range: A program for the analysis of vocabulary in texts [software]. Retrieved from: [URL]
Pellicer-Sánchez, A., & Schmitt, N.
(2010) Incidental Vocabulary Acquisition from an Authentic Novel: Do “Things Fall Apart”? Reading in a Foreign Language, 22 (1), 31–55.Google Scholar
Peters, E.
(2019) The effect of imagery and on-screen text on foreign language vocabulary learning from audiovisual input. TESOL Quarterly, 53 (4), 1008–1032. DOI logoGoogle Scholar
Peters, E., & Webb, S.
(2018) Incidental vocabulary acquisition through viewing L2 television and factors that affect learning. Studies in Second Language Acquisition, 40 (3), 551–77. DOI logoGoogle Scholar
Peters, E., Heynen, E., & Puimège, E.
(2016) Learning Vocabulary through Audiovisual Input: The Differential Effect of L1 Subtitles and Captions. System 63 1, 134–48. DOI logoGoogle Scholar
Pujadas, G.
(2019) Language learning through extensive TV viewing: A study with adolescent EFL learners (Unpublished doctoral dissertation). University of Barcelona.
Pujadas, G., & Muñoz, C.
(2018) What words do we learn better through TV series? The effects of age, proficiency and type of instruction. Paper presented at the Second Language Research Forum, Montreal, Canada.
(2019) Extensive viewing of captioned and subtitled TV series: A study of L2 vocabulary learning by adolescents. The Language Learning Journal, 47 (4), 479–496. DOI logoGoogle Scholar
(2020) Examining Adolescent EFL leaners’ TV viewing comprehension through captions and subtitles. Studies in Second Language Acquisition, 42 (3), 551–575. DOI logoGoogle Scholar
Robinson, P., Mackey, A., Gass, S. M., & Schmidt, R.
(2012) Attention and awareness in second language acquisition. In Gass, S. M., & Mackey, A. (Eds.) The Routledge Handbook of Second Language Acquisition (pp. 247–267).Google Scholar
Rodgers, M. P. H.
(2013) English language learning through viewing television: An investigation of comprehension, incidental vocabulary acquisition, lexical coverage, attitudes, and captions (Unpublished doctoral dissertation). Victoria University of Wellington.
(2016) Extensive listening and viewing: The benefits of audiobooks and television. The European Journal of Applied Linguistics and TEFL, 5 (2), 43–57.Google Scholar
(2018) The images in television programs and the potential for learning unknown words: The relationship between on-screen imagery and vocabulary. ITL – International Journal of Applied Linguistics 169 (1), 191–211. DOI logoGoogle Scholar
Rodgers, M. P. H., & Webb, S.
(2011) Narrow viewing: The vocabulary in related television programs. TESOL Quarterly, 45 (4), 689–717. DOI logoGoogle Scholar
(2020) Incidental vocabulary learning through viewing television. ITL – International Journal of Applied Linguistics, 171 (2), 191–220. DOI logoGoogle Scholar
Suárez, M. D. M., Gilabert, R., & Moskvina, N.
(2021) The mediating role of vocabulary size, working memory, attention and inhibition in early vocabulary learning under different TV genres: An exploratory study. TESOL Journal, 12 (4), e637. DOI logoGoogle Scholar
Sydorenko, T.
(2010) Modality of input and vocabulary acquisition. Language Learning and Technology, 14 (2), 50–73. DOI logoGoogle Scholar
Uchihara, T., Webb, S., & Yanagisawa, A.
(2019) The Effects of Repetition on Incidental Vocabulary Learning: A Meta-Analysis of Correlational Studies. Language Learning, 69 (3), 559–99. DOI logoGoogle Scholar
van Zeeland, H., & Schmitt, N.
(2013) Incidental vocabulary acquisition through L2 listening: A dimensions approach. System, 41 (3), 609–624. DOI logoGoogle Scholar
Vandergrift, L.
(2007) Recent developments in second and foreign language listening comprehension research. Language Teaching, 40 (3), 191–210. DOI logoGoogle Scholar
Vanderplank, R.
(2010) Déjà vu? A decade of research on language laboratories, television and video in language learning. Language Teaching 43 (1), 1–37. DOI logoGoogle Scholar
Vidal, K.
(2011) A comparison of the effects of reading and listening on incidental vocabulary acquisition. Language Learning, 61 (1), 219–258. DOI logoGoogle Scholar
Webb, S.
(2007) The effects of repetition on vocabulary knowledge. Applied Linguistics, 28 (1), 46–65. DOI logoGoogle Scholar
(2010a) Pre-learning low-frequency vocabulary in second language television programmes. Language Teaching Research, 14 (4), 501–15. DOI logoGoogle Scholar
(2010b) Using Glossaries to Increase the Lexical Coverage of Television Programs. Reading in a Foreign Language, 22 (1), 201–221.Google Scholar
Webb, S., & Chang, A. C.
(2015) Second language vocabulary learning through extensive reading with audio support: How do frequency and distribution of occurrence affect learning? Language Teaching Research, 19 (6), 667–686. DOI logoGoogle Scholar
Webb, S., & Nation, P.
(2017) How vocabulary is learned. Oxford University Press.Google Scholar
Webb, S., & Rodgers, M.
(2009a) Vocabulary demands of television programs. Language Learning, 59 (2), 335–66. DOI logoGoogle Scholar
(2009b) The lexical coverage of movies. Applied Linguistics, 30 (3), 407–27. DOI logoGoogle Scholar
Winke, P., Gass, S., & Sydorenko, T.
(2010) The effects of captioning videos used for foreign language listening activities. Language Learning and Technology, 14 (1), 65–86. DOI logoGoogle Scholar
Yang, H. Y.
(2014) The effects of advance organizers and subtitles on EFL learners’ listening comprehension skills. Calico Journal, 31 (3), 345–373. DOI logoGoogle Scholar