This study analyzed the contribution of lexical factors to native-speaking raters’ assessments of comprehensibility and
nativeness in second language (L2) speech. Using transcribed samples to reduce non-lexical sources of bias, 10 naïve L1 English raters
evaluated speech samples from 97 L2 English learners across two tasks (picture description and TOEFL integrated). Subsequently, the 194
transcripts were analyzed through statistical software (e.g., Coh-metrix, VocabProfile) for 29 variables spanning various lexical
dimensions. For the picture description task, separation in lexical correlates of the two constructs was found, with distinct lexical
measures tied to comprehensibility and nativeness. In the TOEFL integrated task, comprehensibility and nativeness were largely
indistinguishable, with identical sets of lexical variables, covering dimensions of diversity and range. Findings are discussed in relation
to the acquisition, assessment, and teaching of lexical properties in L2 speech.
Abercrombie, D. (1949). Teaching pronunciation. ELT Journal, 31, 113–122.
Adams, M. L. (1980). Five co-occurring factors in speaking proficiency. In J. R. Firth (Ed.), Measuring spoken language proficiency (pp. 1–6). Washington, DC: Georgetown University Press.
Anderson, R. C., & Freebody, P. (1981). Vocabulary knowledge. In J. T. Gutherie (Ed.) Comprehension and teaching: Researching reviews (pp. 77–117). Newark, DE. International Reading Association.
Baayen, R. H., Piepenbrock, R., & Gulikers, L. (1995). CELEX. Philadelphia, PA: Linguistic Data Consortium.
Browne, C., Culligan, B., & Phillips, J. (2013). The new general service list. Retrieved from <[URL]>
Crossley, S. A., Cobb, T., & McNamara, D. (2013). Comparing count-based and band-based indices of word frequency: Implications for active vocabulary research and pedagogical applications. System, 411, 965–981.
Crossley, S. A., & McNamara, D. (2013). Applications of text analysis tools for spoken response grading. Language Learning & Technology, 171, 171–192.
Crossley, S. A., Salsbury, T., & McNamara, D. S. (2009). Measuring L2 lexical growth using hypernymic relationships. Language Learning, 591, 307–334.
Crossley, S. A., Salsbury, T., & McNamara, D. S. (2010). The development of polysemy and frequency use in English second language speakers. Language Learning, 601, 573–605.
Crossley, S. A., Salsbury, T., & McNamara, D. S. (2015). Assessing lexical proficiency using analytic ratings: A case for collocation accuracy. Applied Linguistics, 361, 570–590.
Crossley, S. A., Salsbury, T., McNamara, D. S., & Jarvis, S. (2011). What is lexical proficiency? Some answers from computational models of speech data. TESOL Quarterly, 451, 182–193.
Crossley, S. A., Yang, H. S., & McNamara, D. S. (2014). What’s so simple about simplified texts? A computational and psycholinguistic investigation of text comprehension and text processing. Reading in a Foreign Language, 261, 92–113.
Crowther, D., Trofimovich, P., Isaacs, T., & Saito, K. (2015). Does a speaking task affect second language comprehensibility?The Modern Language Journal, 991, 80–95.
Derwing, T. M., & Munro, M. J. (2013). The development of L2 oral language skills in two L1 groups: A seven-year study. Language Learning, 631, 163–185.
Derwing, T. M., Rossiter, M. J., & Ehrensberger-Dow, M. (2002). They speaked and wrote real good: Judgements of non-native and native grammar. Language Awareness, 111, 84–99.
Derwing, T. M., Rossiter, M. J., Munro, M. J., & Thomson, R. I. (2004). L2 fluency: Judgements on different tasks. Language Learning, 541, 655–679.
Field, A. (2009). Discovering statistics using SPSS (3rd ed.). London: Sage.
Ginther, A., Dimova, S., & Yang, R. (2010). Conceptual and empirical relationships between temporal measures of fluency and oral English proficiency with implications for automated scoring. Language Testing, 271, 379–399.
Graesser, A. C., McNamara, D. S., & Kulikowich, J. M. (2011). Coh-Metrix providing multilevel analyses of text characteristics. Educational Researcher, 401, 223–234.
Granena, G. (2014). Individual differences in sequence learning ability and second language acquisition in early childhood and adulthood. Language Learning, 631, 665–703.
Iwashita, N., Brown, A., McNamara, T., & O’Hagan, S. (2008). Assessed levels of second language speaking proficiency: How distinct?Applied Linguistics, 291, 24–49.
Kang, O., Rubin, D., Pickering, L. (2010). Suprasegmental measures of accentedness and judgements of English language learner proficiency in oral English. The Modern Language Journal, 941, 554–566.
Koizumi, R., & In’nami, Y. (2012). Effects of text length on lexical diversity measures: Using short texts with less than 200 tokens. System, 401, 554–564.
Kormos, J., & Dénes, M. (2004). Exploring measures and perceptions of fluency in the speech of second language learners. System, 321, 145–164.
Kuiken, F., & Vedder, I. (2014). Raters’ decisions, rating procedures and rating scales. Language Testing, 311, 279–284.
Kyle, K., & Crossley, S. (2015). Automatically assessing lexical sophistication: Indices, tools, findings, and applications. TESOL Quarterly, 491, 757–786.
Larson-Hall, J. (2010). A guide to doing statistics in second language research using SPSS. New York: Routledge.
Levis, J. (2005). Changing contexts and shifting paradigms in pronunciation teaching. TESOL Quarterly, 391, 369–377.
Lu, X. (2012). The relationship of lexical richness to the quality of ESL learners’ oral narratives. The Modern Language Review, 961, 190–208.
McNamara, D. S., Graesser, A. C., McCarthy, P., & Cai, Z. (2014). Automated evaluation of text and discourse with Coh-Metrix. Cambridge: Cambridge University Press.
McNamara, T. F. (1990). Item response theory and the validation of an ESP test for health professionals. Language Testing, 71, 52–75.
Miller, G. A., Beckwith, R., Fellbaum, C., Gross, D., & Miller, K. J. (1990). Introduction to WordNet: An on-line lexical database. International Journal of Lexicography, 31, 235–244.
Moyer, A. (2013). Foreign accent: The phenomenon of non-native speech. Cambridge: Cambridge University Press.
Munro, M. J., & Derwing, T. M. (2006). The functional load principle in ESL pronunciation instruction: An exploratory study. System, 341, 520–531.
Munro, M. J., & Derwing, T. M. (2009). Comprehensibility as a factor in listener interaction preferences: Implications for the workplace. The Canadian Modern Language Review, 661, 181–202.
Munro, M. J., & Derwing, T. M., Burgess, C. S. (2010). Detection of nonnative speaker status from content-masked speech. Speech Communication, 521, 626–637.
Nation, I. S. P. (2012). The BNC/COCA word family lists (17 September 2012). Unpublished paper. Available at <[URL]>
Neufeld, S., & Billuroğlu, A. (2005). In search of the critical lexical mass: How ‘general’ is the GSL? How ‘academic’ is the AWL? Available at <[URL]>
Oppenheimer, D. M. (2008). The secret life of fluency. Trends in Cognitive Sciences, 121, 237–241.
Perfetti, C. A., Landi, N., & Oakhill, J.The acquisition of reading comprehension skill. In M. J. Snowling & C. Hulme (Eds.), The science of reading: A handbook (pp. 227–247). Oxford: Blackwell.
Plonsky, L., & Oswald, F. L. (2014). How big is “big”? Interpreting effects sizes in L2 research. Language Learning, 641, 878–912.
Read, J. (2004). Plumbing the depths. In P. Bogaards & B. Laufer (Eds.), Vocabulary in second language (pp. 209–227). Amsterdam: John Benjamins.
Révész, A., Michel, M., & Gilabert, R. (2015). Measuring cognitive task demands using dual-task methodology, subjective self-ratings, and expert judgements. Studies in Second Language Acquisition, 38(4), 703–737.
Robinson, P. (2005). Cognitive complexity and task sequencing: Studies in a componential framework for second language task design. International Review of Applied Linguistics in Language Teaching, 431, 1–32.
Saito, K., Trofimovich, P., Isaacs, T. (2016). Second language speech production: Investigating linguistic correlates of comprehensibility and accentedness for learners at different ability levels. Applied Psycholinguistics, 371, 217–240.
Saito, K., Webb, S., Trofimovich, P., & Isaacs, T. (2015). Lexical profiles of comprehensible second language speech. Studies in Second Language Acquisition, 381, 677–701.
Salsbury, T., Crossley, S. A., & McNamara, D. S. (2011). Psycholinguistic word information in second language oral discourse. Second Language Research, 271, 343–360.
Trofimovich, P., & Isaacs, T. (2012). Disentangling accent from comprehensibility. Bilingualism: Language and Cognition, 151, 905–916.
Wen, Q., Wang, L., & Liang, M. (2005). Spoken and written English corpus of Chinese learners. Beijing: Foreign Language Teaching and Research Press.
Wilson, M. D. (1988). The MRC psycholinguistic database: Machine readable dictionary. Behavioural Research Methods, Instruments and Computers, 201, 6–11.
Wray, A. (2002). Formulaic language and the lexicon. New York, NY: Cambridge University Press.
Cited by (10)
Cited by ten other publications
Díez-Ortega, María & Kristopher Kyle
2024. Measuring the development of lexical richness of L2 Spanish: A longitudinal learner corpus study. Studies in Second Language Acquisition 46:1 ► pp. 169 ff.
Tsunemoto, Aki & Pavel Trofimovich
2024. Coherence and comprehensibility in second language speakers’ academic speaking performance. Studies in Second Language Acquisition 46:3 ► pp. 795 ff.
Wang-Taylor, Yixin, Jon Clenton & Yinna Ren
2024. Investigating the Impact of Dialogic and Trialogic Interactive Factors on Chinese Advanced L2 learners’ Vocabulary Use in Spoken Contexts. Languages 9:8 ► pp. 266 ff.
Shao, Yujie, Kazuya Saito & Adam Tierney
2023. How Does Having a Good Ear Promote Instructed Second Language Pronunciation Development? Roles of Domain‐General Auditory Processing in Choral Repetition Training. TESOL Quarterly 57:1 ► pp. 33 ff.
Wang-Taylor, Yixin & Jon Clenton
2022. Exploring the relationship between L2 vocabulary size and academic speaking. System 107 ► pp. 102822 ff.
Saito, Kazuya
2021. What Characterizes Comprehensible and Native‐like Pronunciation Among English‐as‐a‐Second‐Language Speakers? Meta‐Analyses of Phonological, Rater, and Instructional Factors. TESOL Quarterly 55:3 ► pp. 866 ff.
Saito, Kazuya, Yui Suzukida, Mai Tran & Adam Tierney
2021. Domain‐General Auditory Processing Partially Explains Second Language Speech Learning in Classroom Settings: A Review and Generalization Study. Language Learning 71:3 ► pp. 669 ff.
Suzukida, Yui & Kazuya Saito
2021. Which segmental features matter for successful L2 comprehensibility? Revisiting and generalizing the pedagogical value of the functional load principle. Language Teaching Research 25:3 ► pp. 431 ff.
Noreillie, Ann-Sophie, Piet Desmet & Elke Peters
2020. Factors Predicting Low-Intermediate French Learners’ Vocabulary Use in Speaking Tasks. The Canadian Modern Language Review 76:3 ► pp. 194 ff.
2019. HOW DO SECOND LANGUAGE LISTENERS PERCEIVE THE COMPREHENSIBILITY OF FOREIGN-ACCENTED SPEECH?. Studies in Second Language Acquisition 41:5 ► pp. 1133 ff.
This list is based on CrossRef data as of 20 november 2024. Please note that it may not be complete. Sources presented here have been supplied by the respective publishers.
Any errors therein should be reported to them.