Article published in:
ITL - International Journal of Applied Linguistics
Vol. 168:1 (2017) ► pp. 33–69

The updated Vocabulary Levels Test

Developing and validating two new forms of the VLT

| University of Western Ontario
| Kyoto University
| Victoria University of Wellington

The Vocabulary Levels Test (Nation, 1983; Schmitt, Schmitt, & Clapham, 2001) indicates the word frequency level that should be used to select words for learning. The present study involves the development and validation of two new forms of the test. The new forms consist of five levels measuring knowledge of vocabulary at the 1000, 2000, 3000, 4000, and 5000 levels. Items for the tests were sourced from Nation’s (2012) BNC/COCA word lists. The research involved first identifying quality items using the data from 1,463 test takers to create two equivalent forms, and then evaluating the forms with the data from a further 250 test takers. This study also makes an initial attempt to validate the new forms using Messick’s (1989, 1995) validity framework.


This article is currently available as a sample article.

Published online: 05 October 2017
https://doi.org/10.1075/itl.168.1.02web

Full-text

The updated Vocabulary Levels Test
References

References

Andrich, D.
(1988) Rasch models for measurement. Beverly Hills, CA: Sage. CrossrefGoogle Scholar
Bachman, L. F.
(1990) Fundamental considerations in language testing. Oxford: Oxford University Press.Google Scholar
(2000) Modern language testing at the turn of the century: Assuring that what we count counts. Language Testing, 17(1), 1–42. CrossrefGoogle Scholar
Bachman, L. F., & Palmer, A.
(1996) Language testing in practice: Designing and developing useful language tests. Oxford: Oxford University Press.Google Scholar
Beglar, D.
(2010) A Rasch-based validation of the Vocabulary Size Test. Language Testing, 27(1), 101–118. CrossrefGoogle Scholar
Bond, T. G., & Fox, C. M.
(2015) Applying the Rasch model: Fundamental measurement in the human sciences. New York, NY: Routledge.Google Scholar
Chapelle, C. A.
(1999) Validity in language assessment. Annual Review of Applied Linguistics, 19, 254–272. CrossrefGoogle Scholar
Cohen, J.
(1988) Statistical power analysis for the behavioral science (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum Associates.Google Scholar
(1992) A power primer. Psychological Bulletin, 112(1), 155–159. CrossrefGoogle Scholar
Coxhead, A.
(2000) A new academic word list. TESOL Quarterly, 34(2), 213–238. CrossrefGoogle Scholar
Coxhead, A., Nation, I. S. P., & Sim, D.
(2015) Measuring the vocabulary size of native speakers of English in New Zealand secondary schools. New Zealand Journal of Educational Studies, 50(1), 121–135. CrossrefGoogle Scholar
Dang, T. N. Y., & Webb, S.
(2014) The lexical profile of academic spoken English. English for Specific Purposes, 33(1), 66–76. CrossrefGoogle Scholar
(2016) Making an essential word list for beginners. In I. S. P. Nation, Making and using word lists for language learning and testing (pp. 153–167, 188–195). Amsterdam: John Benjamins. CrossrefGoogle Scholar
Hu, H. M., & Nation, P.
(2000) What vocabulary size is needed to read unsimplified texts. Reading in a Foreign Language, 8, 689–696.Google Scholar
Karabatsos, G.
(2000) A critique of Rasch residual fit statistics. Journal of Applied Measurement, 1, 152–176.Google Scholar
Kremmel, B.
(2016) Word families and frequency bands in vocabulary tests: Challenging conventions. TESOL Quarterly, 50(4), 976–987. CrossrefGoogle Scholar
Laufer, B., & Goldstein, Z.
(2004) Testing vocabulary knowledge: Size, strength, and computer adaptiveness. Language Learning, 54(3), 399–436. CrossrefGoogle Scholar
Laufer, B., & Levitzky-Aviad, T.
(2016) Computer Adaptive Test of Size and Strength. Retrieved from http://​catss​.ga/
Laufer, B., & Ravenhorst-Kalovski, G.
(2010) Lexical threshold revisited: Lexical text coverage, learners’ vocabulary size and reading comprehension. Reading in a Foreign Language, 22(1), 15–30.Google Scholar
Linacre, J. M.
(1995) Prioritizing misfit indicators. Rasch Measurement Transactions, 9(2), 422–423.Google Scholar
(2002) What do infit and outfit, mean-square and standardized mean? Rasch Measurement Transactions, 16(2), 878.[ p. 59 ]Google Scholar
(2003) Size vs. significance: Infit and outfit mean-square and standardized chi-square fit statistic. Rasch Measurement Transactions, 17(1), 918.Google Scholar
(2016a) WINSTEPS® Rasch measurement computer program. Beaverton, Oregon: Winsteps​.com.
(2016b) WINSTEPS® Rasch measurement computer programs User’s Guide. Beaverton, Oregon: Winsteps​.com.Google Scholar
Linacre, J. M., & Tennant, A.
(2009) More about critical eigenvalue sizes (variences) in standardized-residual principal components analysis (PCA). Rasch Measurement Transactions, 23(3), 1228.Google Scholar
Mantel, N., & Haenszel, W.
(1959) Statistical aspects of the analysis of data from retrospective studies of disease. Journal of the National Cancer Institute, 22, 719–748.Google Scholar
McLean, S., Kramer, B., & Beglar, D.
(2015) The creation and validation of a listening vocabulay levels test. Language Teaching Research, 19(6), 741–760. CrossrefGoogle Scholar
McNamara, T.
(2006) Validity in language testing: The challenge of Sam Messick’s legacy. Language Assessment Quarterly, 3(1), 31–51. CrossrefGoogle Scholar
Meara, P., & Buxton, B.
(1987) An alternative to multiple choice vocabulary tests.Google Scholar
Meara, P., & Miralpeix, I.
(2017) Tools for researching vocabulary. Bristol, UK: Multilingual Matters.Google Scholar
Messick, S.
(1989) Validity. In R. L. Linn (Ed.), Educational Measurement (3rd ed., pp. 13–103). New York, NY: Macmillan.Google Scholar
(1995) Validity of psychological assessment: Validation of inferences from persons’ responses and performances as scientific inquiry into score meaning. American Psychologist, 50(9), 741–749. CrossrefGoogle Scholar
Nation, I. S. P.
(1983) Testing and teaching vocabulary. Guidelines, 5(1), 12–25.
(2012) The BNC/COCA word family lists. Retrieved from http://​www​.victoria​.ac​.nz​/lals​/about​/staff​/paul​-nation
Nation, I. S. P., & Beglar, D.
(2007) A vocabulary size test. The Language Teacher, 31(7), 9–13.Google Scholar
Nation, I. S. P., & Webb, S.
(2011) Researching and Analyzing Vocabulary. Boston, MA: Heinle.Google Scholar
Raîche, G.
(2005) Critical eigenvalue sizes in standardized residual principal components analysis. Rasch Measurement Transactions, 19(1), 1012.Google Scholar
Rasch, G.
(1960) Probabilistic models for some intelligence and attainment tests. Copenhagen: Danmarks Paedagogiske Institut.Google Scholar
Read, J.
(2000) Assessing Vocabulary. Cambridge: Cambridge University Press. CrossrefGoogle Scholar
Read, J., & Chapelle, C.
(2001) A framework for second language vocabulary assessment. Language Testing, 18(1), 3–32. CrossrefGoogle Scholar
Schmitt, N., Jiang, X., & Grabe, W.
(2011) The percentage of words known in a text and reading comprehension. The Modern Language Journal, 95(1), 26–43. CrossrefGoogle Scholar
Schmitt, N., & Schmitt, D.
(2014) A reassessment of frequency and vocabulary size in L2 vocabulary teaching. Language Teaching, 47(4), 484–503. CrossrefGoogle Scholar
Schmitt, N., Schmitt, D., & Clapham, C.
(2001) Developing and exploring the behaviour of two new versions of the Vocabulary Levels Test. Language Testing, 18(1), 55–88. CrossrefGoogle Scholar
Schmitt, N., & Zimmerman, C.
(2002) Derivative word forms: What do learners know? TESOL Quarterly, 36(2), 145–171. CrossrefGoogle Scholar
Scott, N. W., Fayers, P. M., Aaronson, N. K., Bottomley, A., de Graeff, A., Groenvold, M., … Sprangers, M. A. G.
(2009) A simulation study provided sample size guidance for differential item functioning (DIF) studies using short scales. Journal of Clinical Epidemiology, 62(3), 288–295. Crossref[ p. 60 ]Google Scholar
Smith, A. B., Rush, R., Fallowfield, L. J., Velikova, G., & Sharpe, M.
(2008) Rasch fit statistics and sample size considerations for polytomous data. BMC Medical Research Methodology, 8(33), 1–11.Google Scholar
Smith Jr., E. V.
(2004) Evidence for the reliability of measures and validity of measure interpretation: a Rasch measurement perspective. In E. V. Smith Jr. & R. M. Smith (Eds.), Introduction to Rasch measurement: Theory, models and applications (pp. 93–122). Maple Grove, MN: JAM Press.Google Scholar
Stæhr, L. S.
(2009) Vocabulary knowledge and advanced listening comprehension in English as a foreign language. Studies in Second Language Acquisition, 31(04), 577–607. CrossrefGoogle Scholar
Stevens, J.
(2002) Applied multivariate statistics for the social sciences (4th ed.). Mahwah, NJ: Lawrence Erlbaum Associates.Google Scholar
van Zeeland, H., & Schmitt, N.
(2013) Lexical coverage in L1 and L2 listening comprehension: The same or different from reading comprehension? Applied Linguistics, 34(4), 457–479. CrossrefGoogle Scholar
Webb, S.
(2013) Depth of vocabulary knowledge. In C. Chappelle (Ed.), Encyclopedia of Applied Linguistics (pp. 1656–1663). Oxford, UK: Wiley-Blackwell.Google Scholar
Webb, S., & Sasao, Y.
(2013) New directions in vocabulary testing. RELC Journal, 44(3), 263–278. CrossrefGoogle Scholar
Webb, S. A. & Chang, A. C. -S.
(2012) Second language vocabulary growth. RELC Journal, 43(1), 113–126. CrossrefGoogle Scholar
Webb, S., & Macalister, J.
(2013) Is text written for children appropriate for L2 extensive reading? TESOL Quarterly, 47(2), 300–322. CrossrefGoogle Scholar
Webb, S., & Nation, P.
(2017) How Vocabulary is Learned. Oxford: Oxford University Press.Google Scholar
Webb, S., & Paribakht, T. S.
(2015) What is the relationship between the lexical profile of test items and performance on a standardized English proficiency test? English for Specific Purposes, 38, 34–43. CrossrefGoogle Scholar
Webb, S. & Rodgers, M. P. H.
(2009a) The lexical coverage of movies. Applied Linguistics, 30(3), 407–427. CrossrefGoogle Scholar
(2009b) The vocabulary demands of television programs. Language Learning, 59(2), 335–366. CrossrefGoogle Scholar
Wolfe, E. W., & Smith Jr., E. V.
(2007) Instrument development tools and activities for measure validation using Rasch models: Part 2 – Validation activities. Journal of Applied Measurement, 8, 204–234.Google Scholar
Wright, B. D., & Stone, M. H.
(1979) Best test design. Chicago, IL: MESA Press.Google Scholar
(2004) Making measures. Chicago, IL: Phaneron Press.Google Scholar
Xing, P., & Fulcher, G.
(2007) Reliability assessment for two versions of Vocabulary Levels Tests. System, 35(2), 182–191.Google Scholar
Xue, G., & Nation, I. S. P.
(1984) A university word list. Language Learning and Communication, 3(2), 215229.[ p. 61 ]Google Scholar
Cited by

Cited by other publications

Akbarian, Is’haaq, Fatemeh Farajollahi & Rosa María Jiménez Catalán
2020. EFL learners’ lexical availability: Exploring frequency, exposure, and vocabulary level. System 91 ► pp. 102261 ff. Crossref logo
Ballance, Oliver James & Averil Coxhead
2020. How much vocabulary is needed touse a concordance?. International Journal of Corpus Linguistics 25:1 ► pp. 36 ff. Crossref logo
Busby, Nicole Louise
2020. Words from where?. ITL - International Journal of Applied Linguistics Crossref logo
Dang, Thi Ngoc Yen
2018. The nature of vocabulary in academic speech of hard and soft-sciences. English for Specific Purposes 51 ► pp. 69 ff. Crossref logo
Dang, Thi Ngoc Yen & Stuart Webb
2020.  In Lessons from Good Language Teachers, ► pp. 203 ff. Crossref logo
Durbahn, Marion, Michael Rodgers & Elke Peters
2020. The relationship between vocabulary and viewing comprehension. System 88 ► pp. 102166 ff. Crossref logo
Janebi Enayat, Mostafa, Seyed Mohammad Reza Amirian, Gholamreza Zareian & Saeed Ghaniabadi
2018. Reliable Measure of Written Receptive Vocabulary Size: Using the L2 Depth of Vocabulary Knowledge as a Yardstick. SAGE Open 8:1 ► pp. 215824401775222 ff. Crossref logo
Laufer, Batia & Tom Cobb
2019. How Much Knowledge of Derived Words Is Needed for Reading?. Applied Linguistics Crossref logo
Malone, Jonathan Edward
2019. How Vocabulary is Learned, by S. Webb and P. Nation (2017). Instructed Second Language Acquisition 3:1 ► pp. 115 ff. Crossref logo
Nguyen, Chi-Duc
2020. Lexical Features of Reading Passages in English-language Textbooks for Vietnamese High-school Students: Do they Foster both Content and Vocabulary Gain?. RELC Journal ► pp. 003368821989504 ff. Crossref logo
Pecorari, Diane, Philip Shaw & Hans Malmström
2019. Developing a new academic vocabulary test. Journal of English for Academic Purposes 39 ► pp. 59 ff. Crossref logo
Peters, Elke, Tom Velghe & Tinne Van Rompaey
2019. The VocabLab tests. ITL - International Journal of Applied Linguistics 170:1 ► pp. 53 ff. Crossref logo
Schmitt, Norbert, Paul Nation & Benjamin Kremmel
2020. Moving the field of vocabulary assessment forward: The need for more rigorous test development and validation. Language Teaching 53:1 ► pp. 109 ff. Crossref logo
Stoeckel, Tim, Stuart McLean & Paul Nation
2020. LIMITATIONS OF SIZE AND LEVELS TESTS OF WRITTEN RECEPTIVE VOCABULARY KNOWLEDGE. Studies in Second Language Acquisition ► pp. 1 ff. Crossref logo
Sun, Ye & Thi Ngoc Yen Dang
2020. Vocabulary in high-school EFL textbooks: Texts and learner knowledge. System 93 ► pp. 102279 ff. Crossref logo
Szabo, Csaba Z.
2018. Book review. System 74 ► pp. 220 ff. Crossref logo
Uchihara, Takumi & Jon Clenton
2020. Investigating the role of vocabulary size in second language speaking ability. Language Teaching Research 24:4 ► pp. 540 ff. Crossref logo
Uchihara, Takumi, Stuart Webb & Akifumi Yanagisawa
2019. The Effects of Repetition on Incidental Vocabulary Learning: A Meta‐Analysis of Correlational Studies. Language Learning 69:3 ► pp. 559 ff. Crossref logo
Vu, Duy Van & Elke Peters
2020. Learning Vocabulary from Reading-only, Reading-while-listening, and Reading with Textual Input Enhancement: Insights from Vietnamese EFL Learners. RELC Journal ► pp. 003368822091148 ff. Crossref logo
Webb, Stuart & Paul Nation
2018.  In The Encyclopedia of Applied Linguistics, ► pp. 1 ff. Crossref logo
Webb, Stuart A.
2018.  In The Handbook of Advanced Proficiency in Second Language Acquisition, ► pp. 401 ff. Crossref logo

This list is based on CrossRef data as of 28 august 2020. Please note that it may not be complete. Sources presented here have been supplied by the respective publishers. Any errors therein should be reported to them.