Article published in:
International Journal of Learner Corpus Research
Vol. 6:2 (2020) ► pp. 193–219

Word lists and the role of academic vocabulary use in high stakes speaking assessments

| University of Hawaiʻi at Mānoa
| University of Oregon | Yonsei University
| Georgia State University
The current study explored the extent to which academic vocabulary lists could meet the lexical demands of academic speaking assessments. Indices of word use from lists of academic and general vocabulary were used to predict speaking scores on three TOEFL tasks. The results found weak associations between list-item use and response scores that varied by task. Independent response scores were associated with the use of specialized vocabulary from the first level of the Academic Spoken Word List. Integrated campus situation response scores were most strongly associated with the use of unique words from the Academic Word List. Integrated academic course response scores were associated with the use of more sophisticated general vocabulary. Although the findings provide some support for the use of academic vocabulary lists in speaking assessment preparation, the weak effect sizes point to the need to develop lists of academic vocabulary specific to academic speaking and assessment.
Keywords: academic vocabulary, vocabulary lists, AWL, academic speaking, TOEFL speaking
Article outline
  • 1.Introduction
  • 2.Word lists for vocabulary study
  • 3.Academic vocabulary lists and language production
    • 3.1Academic vocabulary lists and academic writing
    • 3.2Academic vocabulary lists and academic speaking
    • 3.3Academic vocabulary lists and TOEFL speaking
  • 4.Data and methodology
    • 4.1Corpus
    • 4.2Lists of academic vocabulary
    • 4.3Analytical tools and indices
    • 4.4Analytical procedures
  • 5.Results
    • 5.1Independent tasks
    • 5.2Campus situation task
    • 5.3Academic course task
    • 5.4Summary of results
  • 6.Discussion
  • 7.Limitations
  • 8.Conclusion
  • Acknowledgements
  • Note
  • References
Published online: 10 December 2020
https://doi.org/10.1075/ijlcr.20008.smi
References
Ackermann, K., & Chen, Y. H.
(2013) Developing the Academic Collocation List (ACL) – A corpus-driven and expert-judged approach. Journal of English for Academic Purposes, 12(4), 235–247. CrossrefGoogle Scholar
Akaike, H.
(1974) A new look at the statistical model identification. IEEE Transactions on Automatic Control, 191, 716–723. CrossrefGoogle Scholar
Akbarian, I.
(2010) The relationship between vocabulary size and depth for ESP/EAP learners. System, 38(3), 391–401. CrossrefGoogle Scholar
Bauer, L., & Nation, P.
(1993) Word families. International Journal of Lexicography, 6(4), 253–279. CrossrefGoogle Scholar
Berman, R., & Cheng, L.
(2010) English academic language skills: Perceived difficulties by undergraduate and graduate students, and their academic achievement. Canadian Journal of Applied Linguistics, 4(1), 25–40. CrossrefGoogle Scholar
Biber, D., Conrad, S., & Reppen, R.
(1994) Corpus-based approaches to issues in applied linguistics. Applied Linguistics, 15(2), 169–189. CrossrefGoogle Scholar
Biber, D., Conrad, S., & Cortes, V.
(2004) If you look at ... Lexical bundles in university teaching and textbooks. Applied Linguistics, 25(3), 371–405. CrossrefGoogle Scholar
Biber, D., & Gray, B.
(2013) Discourse characteristics of writing and speaking task types on the TOEFL iBT test: A lexico-grammatical analysis (TOEFL iBT™ Report No. RR-13-04). Princeton, NJ: Educational Testing Service. CrossrefGoogle Scholar
Brezina, V., & Gablasova, D.
(2015) Is there a core general vocabulary? Introducing the new general service list. Applied Linguistics, 36(1), 1–22. CrossrefGoogle Scholar
Brooks, L., & Swain, M.
(2014) Contextualizing performances: Comparing performances during TOEFL iBT and real-life academic speaking activities. Language Assessment Quarterly, 11(4), 353–373. CrossrefGoogle Scholar
Brown, A., Iwashita, N., & McNamara, T.
(2005) An examination of rater orientations and test-taker performance on English-for-academic-purposes speaking tasks (TOEFL Monograph No. MS-29). Princeton, NJ: Educational Testing Service. CrossrefGoogle Scholar
Brown, A., & Ducasse, A. M.
(2019) An equal challenge? Comparing TOEFL iBTTM speaking tasks with academic speaking tasks. Language Assessment Quarterly, 16(2), 253–270. CrossrefGoogle Scholar
Caplan, N. A., & Stevens, S. G.
(2017) “Step Out of the Cycle”: Needs, challenges, and successes of international undergraduates at a U.S. University. English for Specific Purposes, 461, 15–28. CrossrefGoogle Scholar
Clark, M. K., & Ishida, S.
(2005) Vocabulary knowledge differences between placed and promoted EAP students. Journal of English for Academic Purposes, 4(3), 225–238. CrossrefGoogle Scholar
Cohen, A. D.
(1988) Statistical power analysis for the behavioral sciences (2nd ed.). Mahwah, NJ: Lawrence Erlbaum Associates.Google Scholar
Coxhead, A.
(2000) A New Academic Word List. TESOL Quarterly, 34(2), 213–238. CrossrefGoogle Scholar
(2011) The academic word list 10 years on: Research and teaching implications. TESOL Quarterly, 45(2), 355–362. CrossrefGoogle Scholar
(2012) Academic vocabulary, writing and English for Academic purposes: Perspectives from second language learners. RELC Journal, 43(1), 137–145. CrossrefGoogle Scholar
(2018) Vocabulary and English for specific purposes: Quantitative and qualitative perspectives. Abingdon: Routledge.Google Scholar
Crossley, S. A., Salsbury, T., McNamara, D. S., & Jarvis, S.
(2011) What is lexical proficiency? Some answers from computational models of speech data. TESOL Quarterly, 45(1), 182–193. CrossrefGoogle Scholar
Crossley, S. A., & McNamara, D. S.
(2013) Applications of text analysis tools for spoken response grading. Language Learning & Technology, 17(2), 171–192.Google Scholar
Cumming, A., Grant, L., Mulcahy-Ernt, P., & Powers, D. E.
(2004) A teacher-verification study of speaking and writing prototype tasks for a new TOEFL. Language Testing, 21(2), 107–145. CrossrefGoogle Scholar
Dang, T. N. Y.
(2018) A Hard Science Spoken Word List. ITL – International Journal of Applied Linguistics, 169(1), 44–71. CrossrefGoogle Scholar
(2020) Vietnamese non-English major EFL university students’ receptive knowledge of the most frequent English words. VNU Journal of Foreign Language Studies, 36(3), 1–11. CrossrefGoogle Scholar
Dang, T. N. Y., Coxhead, A., & Webb, S.
(2017) The academic spoken word list. Language Learning, 67(4), 959–997. CrossrefGoogle Scholar
Dang, T. N. Y., & Webb, S.
(2014) The lexical profile of academic spoken English. English for Specific Purposes, 33(1), 66–76. CrossrefGoogle Scholar
(2016) Making an essential word list for beginners. In I. S. P. Nation (Ed.), Making and using word lists for language learning and testing (pp. 153–167). Amsterdam: John Benjamins. CrossrefGoogle Scholar
Davies, M.
(2008–) The Corpus of Contemporary American English (COCA): One billion words, 1990–2019. Retrieved from https://​www​.english​-corpora​.org​/coca/
Durrant, P.
(2014) Discipline and level specificity in university students’ Written vocabulary. Applied Linguistics, 35(3), 328–356. CrossrefGoogle Scholar
(2016) To what extent is the Academic Vocabulary List relevant to university student writing? English for Specific Purposes, 431, 49–61. CrossrefGoogle Scholar
Eguchi, M., & Kyle, K.
(2020) Continuing to explore the multidimensional nature of lexical sophistication: The case of oral proficiency interviews. Modern Language Journal, 4(2), 381–400. CrossrefGoogle Scholar
Eldridge, J.
(2008) “No, there isn’t an ‘academic vocabulary,’ but…”: A reader responds to K. Hyland and P. Tse’s “Is there an ‘academic vocabulary’?”. TESOL Quarterly, 42(1), 109–113. CrossrefGoogle Scholar
Ellis, N. C., Simpson-Vlach, R., & Maynard, C.
(2008) Formulaic language in native and second-language speakers. TESOL Quarterly, 42(3), 375–396. CrossrefGoogle Scholar
Fan, M.
(2000) How big is the gap and how to narrow it? An investigation into the active and passive vocabulary knowledge of L2 listeners. RELC Journal, 31(2), 105–119. CrossrefGoogle Scholar
Folse, K.
(2011) Applying L2 lexical research findings in ESL teaching. TESOL Quarterly, 45(2), 362–369. CrossrefGoogle Scholar
Gardner, D., & Davies, M.
(2014) A new academic vocabulary list. Applied Linguistics, 35(3), 305–327. CrossrefGoogle Scholar
Goulden, R., Nation, P., & Read, J.
(1990) How large can a receptive vocabulary be? Applied Linguistics, 11(4), 341–363. CrossrefGoogle Scholar
Harsch, C., Ushioda, E., & Ladroue, C.
(2017) Investigating the predictive validity of TOEFL iBT test scores and their use in informing policy in a United Kingdom university setting. ETS Research Report Series 2017(1), 1–80. CrossrefGoogle Scholar
Higginbotham, G., & Reid, J.
(2019) The lexical sophistication of second language learners’ academic essays. Journal of English for Academic Purposes, 371, 127–140. CrossrefGoogle Scholar
Hincks, R.
(2003) Pronouncing the academic word list: Features of L2 student oral presentations. Proceedings of the 15th International Congress of Phonetic Sciences, 1545–1548.Google Scholar
Huang, L. S.
(2010) Seeing eye to eye? The academic writing needs of graduate and undergraduate students from students’ and instructors’ perspectives. Language Teaching Research, 14(4), 517–539. CrossrefGoogle Scholar
Huntley, H.
(2006) Essential academic vocabulary: Mastering the complete academic word list. Boston, MA: Houghton Mifflin Company.Google Scholar
Hyland, K., & Tse, P.
(2007) Is there an “academic vocabulary”? TESOL Quarterly, 41(2), 235–253. CrossrefGoogle Scholar
Iwashita, N., Brown, A., McNamara, T., & Hagan, S. O.
(2008) Assessed levels of second language speaking proficiency: How distinct? Applied Linguistics, 29(1), 24–49. CrossrefGoogle Scholar
Jamieson, J., Eignor, D., Grabe, W., & Kunnan, A.
(2008) Frameworks for a new TOEFL. In C. A. Chapelle, M. K. Enright, & J. M. Jamieson (Eds.), Building a validity argument for the Test of English as a Foreign Language (pp. 55–95). Abingdon: Routledge.Google Scholar
Kyle, K.
(2020) The relationship between features of source text use and integrated writing quality. Assessing Writing, 451, 1–12. CrossrefGoogle Scholar
Kyle, K., & Crossley, S. A.
(2015) Automatically assessing lexical sophistication: Indices, tools, findings, and application. TESOL Quarterly, 49(4), 757–786. CrossrefGoogle Scholar
Kyle, K., Crossley, S. A., & Berger, C.
(2017) The tool for the automatic analysis of lexical sophistication (TAALES): Version 2.0. Behavior Research Methods, 50(3), 1030–1046. CrossrefGoogle Scholar
Kyle, K., Crossley, S. A., & McNamara, D. S.
(2016) Construct validity in TOEFL iBT speaking tasks: Insights from natural language processing. Language Testing, 33(3), 319–340. CrossrefGoogle Scholar
Laufer, B.
(1998) The development of passive and active vocabulary: Same or different? Applied Linguistics, 19 (2), 255–271. CrossrefGoogle Scholar
(2013) Lexical thresholds for reading comprehension: What they are and how they can be used for teaching purposes. TESOL Quarterly, 47(4), 867–872. CrossrefGoogle Scholar
Laufer, B., & Paribakht, T.
(1998) The relationship between passive and active vocabularies: Effects of language learning context. Language Learning, 481, 365–391. CrossrefGoogle Scholar
Laufer, B., & Ravenhorst-Kalvoski, G. C.
(2010) Lexical threshold revisited: Lexical text coverage, learners’ vocabulary size and reading comprehension. Reading in a Foreign Language, 22(1), 15–30.Google Scholar
Lessard-Clouston, M.
(2013) Word lists for vocabulary learning and teaching. The CATESOL Journal, 24(1), 287–304.Google Scholar
Malmström, H., Pecorari, D., & Shaw, P.
(2018) Words for what? Contrasting university students’ receptive and productive academic vocabulary needs. English for Specific Purposes, 501, 28–39. CrossrefGoogle Scholar
Nagy, W., & Anderson, R. C.
(1984) How many words are there in printed school English? Reading Research Quarterly, 19(3), 304–330. CrossrefGoogle Scholar
Nation, I. S. P.
(2001) Learning vocabulary in another language. Cambridge: Cambridge University Press. CrossrefGoogle Scholar
(2006) How large a vocabulary is needed for reading and listening? The Canadian Modern Language Review/La Revue Canadienne Des Langues Vivantes, 63(1), 59–81. CrossrefGoogle Scholar
(2011) Research into practice: Vocabulary. Language Teaching, 44(4), 529–539. CrossrefGoogle Scholar
(2017) The BNC/COCA word family lists (Version 1.0.0) [Data file]. Retrieved from http://​www​.victoria​.ac​.nz​/lals​/staff​/paul​-nation​.aspx
Nation, I. S. P., & Waring, R.
(1997) Vocabulary size, text coverage, and word lists. In N. Schmitt & M. McCarthy (Eds.), Vocabulary: Description, acquisition, and pedagogy (pp. 6–19). Cambridge: Cambridge University Press.Google Scholar
Nation, I. S. P., Coxhead, A., Chung, T. M., & Quero, B.
(2016) Specialized word lists. In I. S. P. Nation (Ed.), Making and using word lists for language learning and testing (pp. 145–151). Amsterdam: John Benjamins. CrossrefGoogle Scholar
Nesi, H., & Gardner, R. C.
(2012) Genres across the disciplines: Student writing in higher education. Cambridge: Cambridge University Press. CrossrefGoogle Scholar
Nesselhauf, N.
(2004) Learner corpora and their potential in language teaching. In J. Sinclair (Ed.), How to use corpora in language teaching (pp. 125–152). Amsterdam: John Benjamins. CrossrefGoogle Scholar
Neufeld, S., Hancioǧlu, N., & Eldridge, J.
(2011) Beware the range in RANGE, and the academic in AWL. System, 39(4), 533–538. CrossrefGoogle Scholar
Ockey, G. J., Koyama, D., Setoguchi, E., & Sun, A.
(2015) The extent to which TOEFL iBT speaking scores are associated with performance on oral language tasks and oral ability components for Japanese university students. Language Testing, 32(1), 39–62. CrossrefGoogle Scholar
Paquot, M.
(2007) Towards a productively-oriented academic word list. In J. Walinski, K. Kredens, & S. Gozdz-Roszkowski (Eds.), Corpora and ICT in language studies. PALC 2005. Lodz studies in LANGUAGE 13 (pp. 127–140). Bern: Peter Lang.Google Scholar
Paribakht, T. S., & Webb, S.
(2016) The relationship between academic vocabulary coverage and scores on a standardized English proficiency test. Journal of English for Academic Purposes, 211, 121–132. CrossrefGoogle Scholar
Plakans, L., & Gebril, A.
(2013) Using multiple texts in an integrated writing assessment: Source text use as a predictor of score. Journal of Second Language Writing, 22(3), 217–230. CrossrefGoogle Scholar
Read, J.
(2000) Assessing vocabulary. Cambridge: Cambridge University Press. CrossrefGoogle Scholar
Römer, U.
(2009) Corpora and language teaching. In A. Lüdeling & M. Kytö (Eds.), Corpus linguistics: An international handbook (pp. 112–131). Berlin: Mouton de Gruyter.Google Scholar
Römer, U., & Donnell, M. B. O.
(2011) From student hard drive to web corpus (part 1): The design, compilation and genre classification of the Michigan Corpus of Upper-level Student Papers (MICUSP). Corpora, 6(2), 159–177. CrossrefGoogle Scholar
RStudio Team
(2020) RStudio: Integrated Development for R. Boston, MA: RStudio, PBC. Retrieved from http://​www​.rstudio​.com/
Saito, K., Webb, S., Trofimovich, P., & Isaacs, T.
(2016) Lexical profiles of comprehensible second language speech: The role of appropriateness, fluency, variation, sophistication, abstractness, and sense relations. Studies in Second Language Acquisition, 38(4), 677–701. CrossrefGoogle Scholar
Schmitt, N.
(2008) Review article: Instructed second language vocabulary learning. Language Teaching Research, 12(3), 329–363. CrossrefGoogle Scholar
Schmitt, N., & Meara, P.
(1997) Researching vocabulary through a word knowledge framework. Studies in Second Language Acquisition, 201, 17–36. CrossrefGoogle Scholar
Schmitt, D., & Schmitt, N.
(2011) Mastering the academic word list. London: Pearson Education.Google Scholar
Schmitt, N., & Zimmerman, C. B.
(2002) Derivative word forms: What do learners know? TESOL Quarterly, 36(2), 145–171. CrossrefGoogle Scholar
Segalowitz, N.
(2007) Access fluidity, attention control, and the acquisition of fluency in a second language. Tesol Quarterly, 41(1), 181–186. CrossrefGoogle Scholar
Simpson, R. C., Briggs, S. L., Ovens, J., & Swales, J. M.
(2002) The Michigan Corpus of Academic Spoken English. Ann Arbor, MI: The Regents of the University of Michigan.Google Scholar
Simpson-Vlach, R., & Ellis, N. C.
(2010) An academic formulas list: New methods in phraseology research. Applied Linguistics, 31(4), 487–512. CrossrefGoogle Scholar
Šišková, Z.
(2016) The relationship between receptive and productive vocabulary of Slavic EFL learners. Topics in Linguistics, 17(2), 26–40. CrossrefGoogle Scholar
Stengers, H., Boers, F., Housen, A., & Eyckmans, J.
(2011) Formulaic sequences and L2 oral proficiency: Does the type of target language influence the association? IRAL – International Review of Applied Linguistics in Language Teaching, 49(4), 321–343. CrossrefGoogle Scholar
van Zeeland, H., & Schmitt, N.
(2013) Lexical coverage in L1 and L2 listening comprehension: The same or different from reading comprehension? Applied Linguistics, 34(4), 457–479. CrossrefGoogle Scholar
Wang, J., Liang, S., & Ge, G.
(2008) Establishment of a medical academic word list. English for Specific Purposes, 271, 442–458. CrossrefGoogle Scholar
Webb, S., & Rodgers, M. P. H.
(2009a) The lexical coverage of movies. Applied Linguistics, 30(3), 407–427. CrossrefGoogle Scholar
(2009b) Vocabulary demands of television programs. Language Learning, 59(2), 335–366. CrossrefGoogle Scholar
West, M.
(1953) A general service list of English words. London: Pearson Longman.Google Scholar
Wray, A.
(2000) Formulaic sequences in second language teaching: Principle and practice. Applied Linguistics, 21(4), 463–489. CrossrefGoogle Scholar
Xi, X.
(2008) Investigating the criterion-related validity of the TOEFL speaking scores for screening and setting standards for ITAs (Research Report No. RR-08-02). Princeton, NJ: Educational Testing Service. CrossrefGoogle Scholar
Zimmerman, C. B.
(2008) Inside reading: The academic word list in context. Oxford: Oxford University Press.Google Scholar