Article published In:
ITL - International Journal of Applied Linguistics: Online-First Articles

The effect of test format on productive recall of derivatives

ORCID logo | University of Western Ontario
ORCID logo | University of Western Ontario

This study investigated the extent to which two recall test formats – contextualized and decontextualized tests – affected productive recall of derivatives, and how the effects of token frequencies of derivatives and L2 receptive vocabulary knowledge on recalling derivatives was moderated by test format. Mixed effects logistic regression models examined the derivatives elicited from L1 (n = 21) and L2 English speakers’ (n = 107) on the two recall tests. Results indicated that contextual cues significantly facilitated recalling derivatives, while such facilitative effects were larger for native speakers and L2 learners with greater vocabulary knowledge. Furthermore, token frequency affected the responses on the decontextualized test to a greater degree compared to the contextualized test. Results suggest that test format influences test-takers’ ability to recall knowledge to produce derivatives.

Keywords: testing vocabulary knowledge, test format effects, frequency effects, productive derivational knowledge, receptive vocabulary knowledge
Article outline
  • Introduction
  • The effect of context on productive recall
  • Measuring derivational knowledge in context and isolation
  • To what extent does context facilitate productive recall of derivatives?
  • How does test format moderate the effects of derivative frequency and prior vocabulary knowledge?
  • Rationale and research questions
  • Method
    • Participants
    • Procedure
    • Derivative form recall test format
      • Contextualized form recall test
      • Decontextualized form recall test
      • Scoring
      • Derivative frequency
      • The Updated Vocabulary Levels Test (uVLT)
  • Analysis
  • Results
    • To what extent does test format moderate productive recall of L1 and L2 derivatives?
    • To what extent does test format affect learners at different vocabulary levels?
    • To what extent does test format moderate the effects of derivative frequency and prior vocabulary knowledge?
  • Discussion
    • To what extent does test format moderate productive recall of derivatives?
    • To what extent does test format affect productive recall of derivatives for learners at different vocabulary levels?
    • To what extent does test format moderate the effects of derivative frequency and prior vocabulary knowledge?
  • Methodological and pedagogical implications
  • Conclusion
  • References
Published online: 12 December 2023
DOI logo
https://doi.org/10.1075/itl.23002.iwa
References
Aviad-Levitzky, T., Laufer, B., & Goldstein, Z.
(2019) The new computer adaptive test of size and strength (CATSS): Development and validation. Language Assessment Quarterly, 16 (3), 345–368. DOI logoGoogle Scholar
Bates, D., Mächler, M., Bolker, B. M., & Walker, S. C.
(2015) Fitting linear mixed-effects models using lme4. Journal of Statistical Software, 67 (1), 1–48. DOI logoGoogle Scholar
Barclay, J. R., Bransford, J. D., Franks, J. J., McCarrell, N. S., & Nitsch, K.
(1974) Comprehension and semantic flexibility. Journal of Verbal Learning and Verbal Behavior, 13 (4), 471–481. DOI logoGoogle Scholar
Berk, R. A.
(2004) Regression Analysis: A Constructive Critique. Sage. DOI logoGoogle Scholar
Brezina, V., & Gablasova, D.
(2015) Is there a core general vocabulary? Introducing the new general service list. Applied Linguistics, 36 (1), 1–22. DOI logoGoogle Scholar
Brysbaert, M., & Stevens, M.
(2018) Power analysis and effect size in mixed effects models: A tutorial. Journal of Cognition, 1 (1):91, 1–20. DOI logoGoogle Scholar
Cheng, L., & DeLuca, C.
(2011) Voices from test-takers: Further evidence for language assessment validation and use, Educational Assessment, 16 (2), 104–122. DOI logoGoogle Scholar
Clahsen, H., & Neubauer, K.
(2020) Morphology, frequency, and the processing of derived words in native and non-native speakers. Lingua, 120(11), 2627–2637. DOI logoGoogle Scholar
Cobb, T., & Laufer, B.
(2021) The nuclear word family list: A list of the most frequent family members, including base and affixed words. Language Learning. 71 (3), 834–871. publication. DOI logoGoogle Scholar
Coxhead, A.
(2000) A new academic word list. TESOL Quarterly, 34 (2), 213–238. DOI logoGoogle Scholar
Coxhead, A., Nation, I. S. P., & Sim, D.
(2015) Measuring the vocabulary size of native speakers of English in New Zealand secondary schools. New Zealand Journal of Educational Studies, 50 (1), 121–135. DOI logoGoogle Scholar
Crossley, S. A., & Salsbury, T.
(2010) Using lexical indices to predict produced and not produced words in second language learners. The Mental Lexicon, 5 (1), 115–147. DOI logoGoogle Scholar
Dang, T. N. Y., & Webb, S.
(2016) Making an essential word list for beginners. In I. S. P. Nation, Making and using word lists for language learning and testing (pp. 153–167, 188–195). John Benjamins. DOI logoGoogle Scholar
Davies, M.
(2009) The 385+ million word Corpus of Contemporary American English (1990– 2008+): Design, architecture, and linguistic insights. International Journal of Corpus Linguistics, 14 (2), 159–190. DOI logoGoogle Scholar
Ellis, N. C.
(2002) Frequency effects in language processing: A review with implications for theories of implicit and explicit language acquisition. Studies in Second Language Acquisition, 24 (2), 143–188. DOI logoGoogle Scholar
Fox, J., & Weisberg, S.
(2019) An R Companion to Applied Regression (3rd ed.). Sage. [URL]
González-Fernández, B., & Schmitt, N.
(2020) Word knowledge: Exploring the relationships and order of acquisition of vocabulary knowledge components. Applied Linguistics, 41 (4) 1–26. DOI logoGoogle Scholar
Gries, S. T.
(2021) (Generalized linear) mixed-effects modeling: A learner corpus example. Language Learning, 71 (3), 757–798. DOI logoGoogle Scholar
Hartig, F.
(2020) DHARMa: Residual diagnostics for hierarchical (multi-level/mixed) regression models. (R package version 0.4.1) [Computer software]. [URL]
Iwaizumi, E. & Webb, S.
(2021) To what extent does productive derivational knowledge of adult L1 speakers and L2 learners at two educational levels differ? TESOL Journal, 12 (4), 1–22. DOI logoGoogle Scholar
Iwaizumi, E., & Webb, S.
(2022) Measuring L1 and L2 productive derivational knowledge: How many derivatives can L1 and L2 learners with differing vocabulary levels produce? TESOL Quarterly, 56 (1), 100–129. DOI logoGoogle Scholar
Iwaizumi, E. & Webb, S.
(2023) To what extent do learner- and word-related variables affect production of derivatives? Language Learning, 73 (1), 301–336. DOI logoGoogle Scholar
Ishii, T., & Schmitt, N.
(2009) Developing an integrated diagnostic test of vocabulary size and depth. RELC Journal, 40 (1), 5–22. DOI logoGoogle Scholar
Kremmel, B.
(2021) Selling the (word) family silver? A response to Webb’s lemma dilemma Studies in Second Language Studies , 43 (5), 962–964. DOI logoGoogle Scholar
Kremmel, B., & Schmitt, N.
(2016) Interpreting vocabulary test scores: What do various item formats tell us about learners’ ability to employ words? Language Assessment Quarterly, 13 (4), 377–392. DOI logoGoogle Scholar
Kyle, K., Crossley, S., & Berger, C.
(2018) The tool for the automatic analysis of lexical sophistication (TAALES): version 2.0. Behavior Research Methods, 50 (3), 1030–1046. DOI logoGoogle Scholar
Laufer, B.
(1998) The development of passive and active vocabulary in a second language: Same or different. Applied Linguistics, 19 (2), 255–271. DOI logoGoogle Scholar
(2023) Understanding L2-derived words in context: Is complete receptive morphological knowledge necessary? Studies in Second Language Acquisition. Advanced online publication. DOI logoGoogle Scholar
Laufer, B., & Aviad-Levitzky, T.
(2017) What type of vocabulary knowledge predicts reading comprehension: Word meaning recall or word meaning recognition? The Modern Language Journal, 101 (4), 729–741. DOI logoGoogle Scholar
Laufer, B., & Goldstein, Z.
(2004) Testing vocabulary knowledge: Size, strength, and computer adaptiveness. Language Learning, 54 (3), 399–436. DOI logoGoogle Scholar
Laufer, B., & Nation, P.
(1995) Vocabulary size and use: Lexical richness in L2 written production. Applied Linguistics, 16 (3), 307–322. DOI logoGoogle Scholar
Laufer, B., & Ravenhorst-Kalovski, G. C.
(2010) Lexical threshold revisited: Lexical text coverage, learners’ vocabulary size and reading comprehension. Reading in a Foreign Language, 22 (1), 15–30. [URL]
Laufer, B., Webb, S., Kim, S., & Yohanan, B.
(2021) How well do learners know derived words in a second language? The effect of proficiency, word frequency and type of affix. ILT – International Journal of Applied Linguistics, 172 (2), 229–258. DOI logoGoogle Scholar
Lenth, R. V.
(2022) emmeans: Estimated marginal means, aka least-squares means. (R package version 1.7.3) [Computer software] [URL]
Leontjev, D., Huhta, A., & Mäntylä, K.
(2016) Word derivational knowledge and writing proficiency: How do they link? System, 59 1, 73–89. DOI logoGoogle Scholar
McCutchen, D., & Stull, S.
(2015) Morphological awareness and children’s writing: accuracy, error, and invention. Reading and Writing, 28 (2), 271–289. DOI logoGoogle Scholar
Miralpeix, I., & Muñoz, C.
(2018) Receptive vocabulary size and its relationship to EFL language skills, International Review of Applied Linguistics in Language Teaching, 56 (1), 1–24. DOI logoGoogle Scholar
Nation, I. S. P.
(2006) How large a vocabulary is needed for reading and listening? The Canadian Modern Language Review, 63 (1), 59–82. DOI logoGoogle Scholar
Nation, Nation, P., & Beglar, D.
(2007) A vocabulary size test. The Language Teacher, 31(7), 9–13. [URL]Google Scholar
Nation, I. S. P.
(2012) The BNC/COCA word family list. [URL]
(2022) Learning vocabulary in another language (2nd ed.). Cambridge University Press. DOI logoGoogle Scholar
Nation, I. S. P., & Webb, S.
(2011) Researching and Analyzing Vocabulary. Heinle ELT.Google Scholar
Peters, E.
(2018) The effect of out-of-class exposure to English language media on learners’ vocabulary knowledge. ITL–International Journal of Applied Linguistics, 169 (1), 142–168. DOI logoGoogle Scholar
Qian, D. D.
(2005) Demystifying lexical inferencing: The role of aspects of vocabulary knowledge. TESL Canada Journal, 22 (2), 34–54. DOI logoGoogle Scholar
(2008) From single words to passages: Contextual effects on predictive power of vocabulary measures for assessing reading performance, Language Assessment Quarterly, 5 (1), 1–19, DOI logoGoogle Scholar
R Core Team
(2022) R: A language and environment for statistical computing. R Foundation for Statistical Computing (Version 4.1.2) [Computer software]. [URL]
Read, J.
(2000) Assessing vocabulary. Cambridge University Press. DOI logoGoogle Scholar
Sasao, Y., & Webb, S.
(2017) The word part levels test. Language Teaching Research, 21 (1), 12–30. DOI logoGoogle Scholar
Schmitt, N.
(2010) Researching vocabulary: A vocabulary research manual. Palgrave Macmillan UK. DOI logoGoogle Scholar
Schmitt, N., & Meara, P.
(1997) Researching vocabulary through a word knowledge framework. Studies in Second Language Acquisition, 19 (1), 17–36. DOI logoGoogle Scholar
Schmitt, N., & Zimmerman, C. B.
(2002) Derivative word forms: What do learners know? TESOL Quarterly, 36 (2), 145–171. DOI logoGoogle Scholar
Snoder, P., & Laufer, B.
(2023) EFL learners’ receptive knowledge of derived words: The case of Swedish Adolescents. TESOL Quarterly, 56 (4), 1242–1265. DOI logoGoogle Scholar
Tulving, E., & Thomson, D. M.
(1973) Encoding specificity and retrieval processes in episodic memory. Psychological Review, 80 (5), 352 – 373. DOI logoGoogle Scholar
Ullman, M. T.
(2001) The neural basis of lexicon and grammar in first and second language: The declarative/procedural model. Bilingualism: Language and Cognition, 4 (1), 105–122. DOI logoGoogle Scholar
Watanabe, Y.
(1997) Input, intake, and retention: Effects of increased processing on incidental learning of foreign language vocabulary. Studies in Second Language Acquisition, 19 (3), 287–308. DOI logoGoogle Scholar
Webb, S.
(2005) Receptive and productive vocabulary learning: The effects of reading and writing on word knowledge. Studies in Second Language Acquisition, 27 (1), 33–52. DOI logoGoogle Scholar
(2008) The effects of context on incidental vocabulary learning. Reading in a Foreign Language, 20 (2), 232–245. [URL]
(2018) Advanced lexical development. In Malovrh, P. A. & A. G. Benati (Eds). The handbook of advanced proficiency in second language acquisition (pp. 401–418). John Wiley & Sons, Inc. DOI logoGoogle Scholar
(2021) The lemma dilemma: How should words be operationalized in research and pedagogy? Studies in Second Language Acquisition, 43 (5), 941–949. DOI logoGoogle Scholar
Webb, S. & Chang, A. C.-S.
(2015) How does prior word knowledge affect vocabulary learning progress in an extensive reading program? Studies in Second Language Acquisition, 43 (5), 941–949. DOI logoGoogle Scholar
Webb, S., Sasao, Y., & Ballance, O.
(2017) The updated vocabulary levels test: Developing and validating two new forms of the VLT. ITL – International Journal of Applied Linguistics, 168 (1), 33–69. DOI logoGoogle Scholar