Article published in:
ITL - International Journal of Applied Linguistics
Vol. 168:1 (2017) ► pp. 33–69

The updated Vocabulary Levels Test

Developing and validating two new forms of the VLT

| University of Western Ontario
| Kyoto University
| Victoria University of Wellington
The Vocabulary Levels Test (Nation, 1983; Schmitt, Schmitt, & Clapham, 2001) indicates the word frequency level that should be used to select words for learning. The present study involves the development and validation of two new forms of the test. The new forms consist of five levels measuring knowledge of vocabulary at the 1000, 2000, 3000, 4000, and 5000 levels. Items for the tests were sourced from Nation’s (2012) BNC/COCA word lists. The research involved first identifying quality items using the data from 1,463 test takers to create two equivalent forms, and then evaluating the forms with the data from a further 250 test takers. This study also makes an initial attempt to validate the new forms using Messick’s (1989, 1995) validity framework.


This article is currently available as a sample article.

Published online: 05 October 2017
https://doi.org/10.1075/itl.168.1.02web

Full-text

The updated Vocabulary Levels Test
References

References

Andrich, D.
(1988) Rasch models for measurement. Beverly Hills, CA: Sage. CrossrefGoogle Scholar
Bachman, L. F.
(1990) Fundamental considerations in language testing. Oxford: Oxford University Press.Google Scholar
(2000) Modern language testing at the turn of the century: Assuring that what we count counts. Language Testing, 17(1), 1–42. CrossrefGoogle Scholar
Bachman, L. F., & Palmer, A.
(1996) Language testing in practice: Designing and developing useful language tests. Oxford: Oxford University Press.Google Scholar
Beglar, D.
(2010) A Rasch-based validation of the Vocabulary Size Test. Language Testing, 27(1), 101–118. CrossrefGoogle Scholar
Bond, T. G., & Fox, C. M.
(2015) Applying the Rasch model: Fundamental measurement in the human sciences. New York, NY: Routledge.Google Scholar
Chapelle, C. A.
(1999) Validity in language assessment. Annual Review of Applied Linguistics, 19, 254–272. CrossrefGoogle Scholar
Cohen, J.
(1988) Statistical power analysis for the behavioral science (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum Associates.Google Scholar
(1992) A power primer. Psychological Bulletin, 112(1), 155–159. CrossrefGoogle Scholar
Coxhead, A.
(2000) A new academic word list. TESOL Quarterly, 34(2), 213–238. CrossrefGoogle Scholar
Coxhead, A., Nation, I. S. P., & Sim, D.
(2015) Measuring the vocabulary size of native speakers of English in New Zealand secondary schools. New Zealand Journal of Educational Studies, 50(1), 121–135. CrossrefGoogle Scholar
Dang, T. N. Y., & Webb, S.
(2014) The lexical profile of academic spoken English. English for Specific Purposes, 33(1), 66–76. CrossrefGoogle Scholar
(2016) Making an essential word list for beginners. In I. S. P. Nation, Making and using word lists for language learning and testing (pp. 153–167, 188–195). Amsterdam: John Benjamins. CrossrefGoogle Scholar
Hu, H. M., & Nation, P.
(2000) What vocabulary size is needed to read unsimplified texts. Reading in a Foreign Language, 8, 689–696.Google Scholar
Karabatsos, G.
(2000) A critique of Rasch residual fit statistics. Journal of Applied Measurement, 1, 152–176.Google Scholar
Kremmel, B.
(2016) Word families and frequency bands in vocabulary tests: Challenging conventions. TESOL Quarterly, 50(4), 976–987. CrossrefGoogle Scholar
Laufer, B., & Goldstein, Z.
(2004) Testing vocabulary knowledge: Size, strength, and computer adaptiveness. Language Learning, 54(3), 399–436. CrossrefGoogle Scholar
Laufer, B., & Levitzky-Aviad, T.
(2016) Computer Adaptive Test of Size and Strength. Retrieved from http://​catss​.ga/
Laufer, B., & Ravenhorst-Kalovski, G.
(2010) Lexical threshold revisited: Lexical text coverage, learners’ vocabulary size and reading comprehension. Reading in a Foreign Language, 22(1), 15–30.Google Scholar
Linacre, J. M.
(1995) Prioritizing misfit indicators. Rasch Measurement Transactions, 9(2), 422–423.Google Scholar
(2002) What do infit and outfit, mean-square and standardized mean? Rasch Measurement Transactions, 16(2), 878.[ p. 59 ]Google Scholar
(2003) Size vs. significance: Infit and outfit mean-square and standardized chi-square fit statistic. Rasch Measurement Transactions, 17(1), 918.Google Scholar
(2016a) WINSTEPS® Rasch measurement computer program. Beaverton, Oregon: Winsteps​.com.
(2016b) WINSTEPS® Rasch measurement computer programs User’s Guide. Beaverton, Oregon: Winsteps​.com.Google Scholar
Linacre, J. M., & Tennant, A.
(2009) More about critical eigenvalue sizes (variences) in standardized-residual principal components analysis (PCA). Rasch Measurement Transactions, 23(3), 1228.Google Scholar
Mantel, N., & Haenszel, W.
(1959) Statistical aspects of the analysis of data from retrospective studies of disease. Journal of the National Cancer Institute, 22, 719–748.Google Scholar
McLean, S., Kramer, B., & Beglar, D.
(2015) The creation and validation of a listening vocabulay levels test. Language Teaching Research, 19(6), 741–760. CrossrefGoogle Scholar
McNamara, T.
(2006) Validity in language testing: The challenge of Sam Messick’s legacy. Language Assessment Quarterly, 3(1), 31–51. CrossrefGoogle Scholar
Meara, P., & Buxton, B.
(1987) An alternative to multiple choice vocabulary tests.Google Scholar
Meara, P., & Miralpeix, I.
(2017) Tools for researching vocabulary. Bristol, UK: Multilingual Matters.Google Scholar
Messick, S.
(1989) Validity. In R. L. Linn (Ed.), Educational Measurement (3rd ed., pp. 13–103). New York, NY: Macmillan.Google Scholar
(1995) Validity of psychological assessment: Validation of inferences from persons’ responses and performances as scientific inquiry into score meaning. American Psychologist, 50(9), 741–749. CrossrefGoogle Scholar
Nation, I. S. P.
(1983) Testing and teaching vocabulary. Guidelines, 5(1), 12–25.
(2012) The BNC/COCA word family lists. Retrieved from http://​www​.victoria​.ac​.nz​/lals​/about​/staff​/paul​-nation
Nation, I. S. P., & Beglar, D.
(2007) A vocabulary size test. The Language Teacher, 31(7), 9–13.Google Scholar
Nation, I. S. P., & Webb, S.
(2011) Researching and Analyzing Vocabulary. Boston, MA: Heinle.Google Scholar
Raîche, G.
(2005) Critical eigenvalue sizes in standardized residual principal components analysis. Rasch Measurement Transactions, 19(1), 1012.Google Scholar
Rasch, G.
(1960) Probabilistic models for some intelligence and attainment tests. Copenhagen: Danmarks Paedagogiske Institut.Google Scholar
Read, J.
(2000) Assessing Vocabulary. Cambridge: Cambridge University Press. CrossrefGoogle Scholar
Read, J., & Chapelle, C.
(2001) A framework for second language vocabulary assessment. Language Testing, 18(1), 3–32. CrossrefGoogle Scholar
Schmitt, N., Jiang, X., & Grabe, W.
(2011) The percentage of words known in a text and reading comprehension. The Modern Language Journal, 95(1), 26–43. CrossrefGoogle Scholar
Schmitt, N., & Schmitt, D.
(2014) A reassessment of frequency and vocabulary size in L2 vocabulary teaching. Language Teaching, 47(4), 484–503. CrossrefGoogle Scholar
Schmitt, N., Schmitt, D., & Clapham, C.
(2001) Developing and exploring the behaviour of two new versions of the Vocabulary Levels Test. Language Testing, 18(1), 55–88. CrossrefGoogle Scholar
Schmitt, N., & Zimmerman, C.
(2002) Derivative word forms: What do learners know? TESOL Quarterly, 36(2), 145–171. CrossrefGoogle Scholar
Scott, N. W., Fayers, P. M., Aaronson, N. K., Bottomley, A., de Graeff, A., Groenvold, M., … Sprangers, M. A. G.
(2009) A simulation study provided sample size guidance for differential item functioning (DIF) studies using short scales. Journal of Clinical Epidemiology, 62(3), 288–295. Crossref[ p. 60 ]Google Scholar
Smith, A. B., Rush, R., Fallowfield, L. J., Velikova, G., & Sharpe, M.
(2008) Rasch fit statistics and sample size considerations for polytomous data. BMC Medical Research Methodology, 8(33), 1–11.Google Scholar
Smith Jr., E. V.
(2004) Evidence for the reliability of measures and validity of measure interpretation: a Rasch measurement perspective. In E. V. Smith Jr. & R. M. Smith (Eds.), Introduction to Rasch measurement: Theory, models and applications (pp. 93–122). Maple Grove, MN: JAM Press.Google Scholar
Stæhr, L. S.
(2009) Vocabulary knowledge and advanced listening comprehension in English as a foreign language. Studies in Second Language Acquisition, 31(04), 577–607. CrossrefGoogle Scholar
Stevens, J.
(2002) Applied multivariate statistics for the social sciences (4th ed.). Mahwah, NJ: Lawrence Erlbaum Associates.Google Scholar
van Zeeland, H., & Schmitt, N.
(2013) Lexical coverage in L1 and L2 listening comprehension: The same or different from reading comprehension? Applied Linguistics, 34(4), 457–479. CrossrefGoogle Scholar
Webb, S.
(2013) Depth of vocabulary knowledge. In C. Chappelle (Ed.), Encyclopedia of Applied Linguistics (pp. 1656–1663). Oxford, UK: Wiley-Blackwell.Google Scholar
Webb, S., & Sasao, Y.
(2013) New directions in vocabulary testing. RELC Journal, 44(3), 263–278. CrossrefGoogle Scholar
Webb, S. A. & Chang, A. C. -S.
(2012) Second language vocabulary growth. RELC Journal, 43(1), 113–126. CrossrefGoogle Scholar
Webb, S., & Macalister, J.
(2013) Is text written for children appropriate for L2 extensive reading? TESOL Quarterly, 47(2), 300–322. CrossrefGoogle Scholar
Webb, S., & Nation, P.
(2017) How Vocabulary is Learned. Oxford: Oxford University Press.Google Scholar
Webb, S., & Paribakht, T. S.
(2015) What is the relationship between the lexical profile of test items and performance on a standardized English proficiency test? English for Specific Purposes, 38, 34–43. CrossrefGoogle Scholar
Webb, S. & Rodgers, M. P. H.
(2009a) The lexical coverage of movies. Applied Linguistics, 30(3), 407–427. CrossrefGoogle Scholar
(2009b) The vocabulary demands of television programs. Language Learning, 59(2), 335–366. CrossrefGoogle Scholar
Wolfe, E. W., & Smith Jr., E. V.
(2007) Instrument development tools and activities for measure validation using Rasch models: Part 2 – Validation activities. Journal of Applied Measurement, 8, 204–234.Google Scholar
Wright, B. D., & Stone, M. H.
(1979) Best test design. Chicago, IL: MESA Press.Google Scholar
(2004) Making measures. Chicago, IL: Phaneron Press.Google Scholar
Xing, P., & Fulcher, G.
(2007) Reliability assessment for two versions of Vocabulary Levels Tests. System, 35(2), 182–191.Google Scholar
Xue, G., & Nation, I. S. P.
(1984) A university word list. Language Learning and Communication, 3(2), 215229.[ p. 61 ]Google Scholar
Cited by

Cited by 44 other publications

Ahrabi Fakhr, Masoud, Davood Borzabadi Farahani & Ali Akbar Khomeijani Farahani
2021. Incidental Vocabulary Learning and Retention from Audiovisual Input and Factors Affecting Them. English Teaching & Learning 45:2 ► pp. 167 ff. Crossref logo
Akbarian, Is’haaq, Fatemeh Farajollahi & Rosa María Jiménez Catalán
2020. EFL learners’ lexical availability: Exploring frequency, exposure, and vocabulary level. System 91 ► pp. 102261 ff. Crossref logo
Ballance, Oliver James & Averil Coxhead
2020. How much vocabulary is needed touse a concordance?. International Journal of Corpus Linguistics 25:1 ► pp. 36 ff. Crossref logo
Boers, Frank
2021. Glossing and vocabulary learning. Language Teaching ► pp. 1 ff. Crossref logo
Busby, Nicole Louise
2021. Words from where?. ITL - International Journal of Applied Linguistics 172:1 ► pp. 58 ff. Crossref logo
Cobb, Tom & Batia Laufer
2021. The Nuclear Word Family List: A List of the Most Frequent Family Members, Including Base and Affixed Words. Language Learning 71:3 ► pp. 834 ff. Crossref logo
Dang, Thi Ngoc Yen
2018. The nature of vocabulary in academic speech of hard and soft-sciences. English for Specific Purposes 51 ► pp. 69 ff. Crossref logo
Dang, Thi Ngoc Yen, Cailing Lu & Stuart Webb
2021. INCIDENTAL LEARNING OF SINGLE WORDS AND COLLOCATIONS THROUGH VIEWING AN ACADEMIC LECTURE. Studies in Second Language Acquisition ► pp. 1 ff. Crossref logo
Dang, Thi Ngoc Yen & Stuart Webb
2020.  In Lessons from Good Language Teachers, ► pp. 203 ff. Crossref logo
De la Maya Retamar, Guadalupe & Magdalena López-Pérez
2021. Hábitos lectores y competencia léxica de futuros profesores de inglés em Educación Primaria. Educação Formação 6:1 ► pp. e3504 ff. Crossref logo
Durbahn, Marion, Michael Rodgers & Elke Peters
2020. The relationship between vocabulary and viewing comprehension. System 88 ► pp. 102166 ff. Crossref logo
El-Dakhs, Dina Abdel Salam, Suhad Sonbul & Rawan Alwazzan
2021. Learning phrasal verbs in the EFL classroom: the effect of prior vocabulary knowledge and opacity. International Review of Applied Linguistics in Language Teaching 0:0 Crossref logo
Ha, Hung Tan
2021. A Rasch-based validation of the Vietnamese version of the Listening Vocabulary Levels Test. Language Testing in Asia 11:1 Crossref logo
Iwaizumi, Emi & Stuart Webb
2021. Measuring L1 and L2 Productive Derivational Knowledge: How Many Derivatives Can L1 and L2 Learners with Differing Vocabulary Levels Produce?. TESOL Quarterly Crossref logo
Janebi Enayat, Mostafa, Seyed Mohammad Reza Amirian, Gholamreza Zareian & Saeed Ghaniabadi
2018. Reliable Measure of Written Receptive Vocabulary Size: Using the L2 Depth of Vocabulary Knowledge as a Yardstick. SAGE Open 8:1 ► pp. 215824401775222 ff. Crossref logo
Janebi Enayat, Mostafa & Ali Derakhshan
2021. Vocabulary size and depth as predictors of second language speaking ability. System 99 ► pp. 102521 ff. Crossref logo
Laufer, Batia & Tom Cobb
2020. How Much Knowledge of Derived Words Is Needed for Reading?. Applied Linguistics 41:6 ► pp. 971 ff. Crossref logo
Laufer, Batia, Stuart Webb, Su Kyung Kim & Beverley Yohanan
2021. How well do learners know derived words in a second language?. ITL - International Journal of Applied Linguistics Crossref logo
Liu, Yeu-Ting, Hossein Nassaji & Wen-Ta Tseng
2021. Effects of internal and external attentional manipulations and working memory on second language vocabulary learning. Language Teaching Research ► pp. 136216882110301 ff. Crossref logo
Mairano, Paolo & Fabian Santiago
2020. What vocabulary size tells us about pronunciation skills: Issues in assessing L2 learners. Journal of French Language Studies 30:2 ► pp. 141 ff. Crossref logo
Malone, Jonathan Edward
2019. How Vocabulary is Learned, by S. Webb and P. Nation (2017). Instructed Second Language Acquisition 3:1 ► pp. 115 ff. Crossref logo
Nation, I.S.P. & Averil Coxhead
2021.  In Measuring Native-Speaker Vocabulary Size, Crossref logo
Nation, Paul
2020. Is it worth teaching vocabulary?. TESOL Journal Crossref logo
Nguyen, Chi-Duc
2020. Lexical Features of Reading Passages in English-language Textbooks for Vietnamese High-school Students: Do they Foster both Content and Vocabulary Gain?. RELC Journal ► pp. 003368821989504 ff. Crossref logo
Papin, Kevin & Regina Kaplan-Rakowski
2020. A Study of Vocabulary Learning Using Annotated 360° Pictures. SSRN Electronic Journal Crossref logo
Pecorari, Diane, Philip Shaw & Hans Malmström
2019. Developing a new academic vocabulary test. Journal of English for Academic Purposes 39 ► pp. 59 ff. Crossref logo
Peters, Elke, Tom Velghe & Tinne Van Rompaey
2019. The VocabLab tests. ITL - International Journal of Applied Linguistics 170:1 ► pp. 53 ff. Crossref logo
Schmitt, Norbert, Paul Nation & Benjamin Kremmel
2020. Moving the field of vocabulary assessment forward: The need for more rigorous test development and validation. Language Teaching 53:1 ► pp. 109 ff. Crossref logo
Schmitt, Norbert & Diane Schmitt
2020.  In Vocabulary in Language Teaching, Crossref logo
Sonbul, Suhad, Dina Abdel Salam El-Dakhs & Hind Al-Otaibi
2020. Productive versus receptive L2 knowledge of polysemous phrasal verbs: A comparison of determining factors. System 95 ► pp. 102361 ff. Crossref logo
Stewart, Jeffrey, Tim Stoeckel, Stuart McLean, Paul Nation & Geoffrey G. Pinchbeck
2021. WHAT THE RESEARCH SHOWS ABOUT WRITTEN RECEPTIVE VOCABULARY TESTING. Studies in Second Language Acquisition 43:2 ► pp. 462 ff. Crossref logo
Stoeckel, Tim, Stuart McLean & Paul Nation
2021. LIMITATIONS OF SIZE AND LEVELS TESTS OF WRITTEN RECEPTIVE VOCABULARY KNOWLEDGE. Studies in Second Language Acquisition 43:1 ► pp. 181 ff. Crossref logo
Sun, Ye & Thi Ngoc Yen Dang
2020. Vocabulary in high-school EFL textbooks: Texts and learner knowledge. System 93 ► pp. 102279 ff. Crossref logo
Szabo, Csaba Z.
2018. Book review. System 74 ► pp. 220 ff. Crossref logo
Uchihara, Takumi & Jon Clenton
2020. Investigating the role of vocabulary size in second language speaking ability. Language Teaching Research 24:4 ► pp. 540 ff. Crossref logo
Uchihara, Takumi, Stuart Webb, Kazuya Saito & Pavel Trofimovich
2021. THE EFFECTS OF TALKER VARIABILITY AND FREQUENCY OF EXPOSURE ON THE ACQUISITION OF SPOKEN WORD KNOWLEDGE. Studies in Second Language Acquisition ► pp. 1 ff. Crossref logo
Uchihara, Takumi, Stuart Webb & Akifumi Yanagisawa
2019. The Effects of Repetition on Incidental Vocabulary Learning: A Meta‐Analysis of Correlational Studies. Language Learning 69:3 ► pp. 559 ff. Crossref logo
Van Vu, Duy & Elke Peters
2021. INCIDENTAL LEARNING OF COLLOCATIONS FROM MEANINGFUL INPUT. Studies in Second Language Acquisition ► pp. 1 ff. Crossref logo
Vu, Duy Van & Elke Peters
2020. Learning Vocabulary from Reading-only, Reading-while-listening, and Reading with Textual Input Enhancement: Insights from Vietnamese EFL Learners. RELC Journal ► pp. 003368822091148 ff. Crossref logo
Webb, Stuart
2021. A DIFFERENT PERSPECTIVE ON THE LIMITATIONS OF SIZE AND LEVELS TESTS OF WRITTEN RECEPTIVE VOCABULARY KNOWLEDGE. Studies in Second Language Acquisition 43:2 ► pp. 454 ff. Crossref logo
Webb, Stuart & Paul Nation
2018.  In The Encyclopedia of Applied Linguistics, ► pp. 1 ff. Crossref logo
Webb, Stuart A.
2018.  In The Handbook of Advanced Proficiency in Second Language Acquisition, ► pp. 401 ff. Crossref logo
Xodabande, Ismail, Asqar Pourhassan & Mohammadreza Valizadeh
2021. Self-directed learning of core vocabulary in English by EFL learners: comparing the outcomes from paper and mobile application flashcards. Journal of Computers in Education Crossref logo
Yang, Lu & Averil Coxhead
2020. A Corpus-based Study of Vocabulary in the New Concept English Textbook Series. RELC Journal ► pp. 003368822096416 ff. Crossref logo

This list is based on CrossRef data as of 14 september 2021. Please note that it may not be complete. Sources presented here have been supplied by the respective publishers. Any errors therein should be reported to them.