Cover not available
Part of
Meaningful Language Test Scores: Research to enhance score interpretation
Edited by Spiros Papageorgiou and Venessa F. Manna
[Innovations in Language Learning and Assessment 1] 2023
► pp. 120–140

Chapter 7
Mapping TOEFL® Essentials™ speaking and writing scores to the CEFR levels

| Educational Testing Service
| Educational Testing Service
| Educational Testing Service
| Educational Testing Service
This chapter describes a study to map scores from a new English language proficiency assessment, the TOEFL® Essentials™ test, to the Common European Framework of Reference (CEFR). The project started with a comparison of content between the TOEFL Essentials test and the CEFR descriptors, which established a claim that scores were relevant to language ability as described in the CEFR. A standard setting study, using the Performance Profile method, was then conducted to establish cut scores for each CEFR level. Additional information to support the reasonableness of these cut scores is also reported.
Article outline
  • Introduction
  • Speaking and Writing sections of the TOEFL Essentials test
  • Methodology
    • Construct congruence with the CEFR
    • Materials used for the standard setting meeting
    • Panelist selection and preparation
    • Standard setting procedure
    • Online context
  • Results
    • Construct congruence
    • Raw cut scores for CEFR levels
    • Conversion of raw cut scores to band scores
  • Discussion
    • Validity of study results
    • Advantages and disadvantages of working remotely
    • Limitations and future research
  • Conclusion
  • References
Published online: 29 June 2023
DOI logo
https://doi.org/10.1075/illa.1.07dav
References (18)
References
Cizek, G. J., & Bunch, M. B. (2007). Standard setting: A guide to establishing performance standards on tests. Sage. DOI logoGoogle Scholar
Council of Europe. (n.d.). CEFR Descriptors (searchable). Retrieved on 8 February 2023 from [URL]
. (2001). The Common European Framework of Reference for languages: Learning, teaching, assessment. Cambridge University Press. Retrieved on 8 February 2023 from [URL]
. (2009). Relating language examinations to the Common European Framework of Reference for Languages: Learning, teaching, assessment. A manual. Council of Europe. Retrieved on 8 February 2023 from [URL]
. (2020). Common European Framework of Reference for Languages: Learning, teaching, assessment. Companion volume with new descriptors. Retrieved on 8 February 2023 from [URL]
ETS. (2021a). TOEFL Essentials Test: Speaking Scoring Guide. Retrieved on 8 February 2023 from [URL]
. (2021b). TOEFL Essentials Test: Writing Scoring Guide. Retrieved on 8 February 2023 from [URL]
. (2021c). Setting Score Requirements. Retrieved on 8 February 2023 from [URL]
Fleckenstein, J., Keller, S., Kruger, M., Tannenbaum, R. J., & Koller, O. (2020). Linking TOEFL iBT® writing rubrics to CEFR levels: Cut scores and validity evidence from a standard setting study. Assessing Writing, 43, 1–15. DOI logoGoogle Scholar
Geisinger, K. F., & McCormick, C. A. (2010). Adopting cut scores: Post-standard-setting panel considerations for decision makers. Educational Measurement: Issues and Practice, 29(1), 38–44. DOI logoGoogle Scholar
Gu, L., Li, Y., Monfils, L., & Papageorgiou, S. (this volume). Statistical methodology for developing vertical scales for language tests. In S. Papageorgiou & V. F. Manna (Eds.), Meaningful language test scores: Research to enhance score interpretation. John Benjamins.
Livingston, S. A. (2014). Equating test scores (without IRT). ETS. Retrieved on 8 February 2023 from [URL]Google Scholar
Papageorgiou, S. (2010). Investigating the decision-making process of standard setting participants. Language Testing, 27(2), 261–82. DOI logoGoogle Scholar
Papageorgiou, S., Davis, L., Norris, J. M., Garcia Gomez, P., Manna, V. F., & Monfils, L. (2021). Design framework for the TOEFL® Essentials™ test 2021 (Research Memorandum No. RM–21–03). ETS. Retrieved on 8 February 2023 from [URL]Google Scholar
Papageorgiou, S., Ginsburgh, M., & Garcia Gomez, P. (this volume). Assessment design issues in developing vertical scales for language tests. In S. Papageorgiou & V. F. Manna (Eds.), Meaningful language test scores: Research to enhance score interpretation. John Benjamins.
Papageorgiou, S., Wu, S., Hsieh, C.-N., Tannenbaum, R. J., & Cheng, M. (2019). Mapping the TOEFL iBT test scores to China’s Standards of English Language Ability: Implications for score interpretation and use (Research Report No. TOEFL-RR–89). ETS. DOI logoGoogle Scholar
Schmidgall, J. (this volume). Exploring the benefits and challenges of web-based approaches to standard setting. In S. Papageorgiou & V. F. Manna (Eds.), Meaningful language test scores: Research to enhance score interpretation. John Benjamins.
Tannenbaum, R. J., & Cho, Y. (2014). Critical factors to consider in evaluating standard-setting studies to map language test scores to frameworks of language proficiency. Language Assessment Quarterly, 11(3), 233–249. DOI logoGoogle Scholar