Cover not available
Part of
Meaningful Language Test Scores: Research to enhance score interpretation
Edited by Spiros Papageorgiou and Venessa F. Manna
[Innovations in Language Learning and Assessment 1] 2023
► pp. 35–60

Chapter 3
Assessment design issues in developing vertical scales for language tests

| Educational Testing Service
| Educational Testing Service
| Educational Testing Service

To support decisions about readiness to take a test in the TOEFL® Family of Assessments, a multi-year project was launched in 2017 to explore the possibility of expressing scores across multiple tests on a single, consistently interpreted scale by applying vertical linking procedures. This chapter focuses on the assessment design aspects of the vertical linking project. It describes the selection of vertical linking items for listening and reading and the development of test forms so that test taker responses could be collected. The chapter also discusses examples of content analysis of the vertical linking items flagged during the statistical analysis described in Chapter 4. The chapter concludes with implications for suites of language proficiency tests regarding the selection of vertical linking items.

Article outline
  • Introduction
  • Overview of the vertical linking design for the TOEFL family of assessments
  • Selection of listening and reading task types
  • Selection of vertical linking items for the test forms
    • TOEFL Primary test forms with TOEFL Junior vertical linking items
    • TOEFL Junior test forms with TOEFL Primary vertical linking items
    • TOEFL Junior test forms with TOEFL ITP vertical linking items
    • TOEFL ITP test forms with TOEFL Junior vertical linking items
    • TOEFL ITP test forms with TOEFL iBT vertical linking items
    • TOEFL iBT test forms with TOEFL ITP vertical linking items
  • Content analysis after data collection
  • Conclusion
  • Acknowledgments
  • Notes
  • References
Published online: 29 June 2023
DOI logo
https://doi.org/10.1075/illa.1.03pap
References
Council of Europe
(2001) Common European Framework of Reference for Languages: Learning, teaching, assessment. Cambridge University Press.Google Scholar
Cho, Y., Ginsburgh, M., Moulder, B., Morgan, R., Xi, X., & Hauck, M.
(2016) Designing the TOEFL Primary Tests (ETS Research Memorandum RM–16–02). ETS. Retrieved on 7 February 2023 from [URL]Google Scholar
ETS
(2017) Test taker handbook for the TOEFL ITP® tests. Retrieved on 7 February 2023 from [URL]
(2018) Handbook for the TOEFL Junior® tests. Retrieved on 7 February 2023 from [URL]
(2019) Handbook for the TOEFL Primary® tests. Retrieved on 7 February 2023 from [URL]
Gu, L., Li, Y., Monfils, L., & Papageorgiou, S.
this volume). Statistical methodology for developing vertical scales for language tests. In S. Papageorgiou & V. F. Manna Eds. Meaningful language test scores: Research to enhance score interpretation John Benjamins
Gu., L., Wang, L., Cho, Y.
(2020) Impact of printed stems on listening performance of TOEFL ITP test takers (Unpublished technical report). ETS.Google Scholar
Kolen, M. J., & Brennan, R. L.
(2014) Test equating, scaling, and linking: Methods and practices (3rd ed.). Springer. DOI logoGoogle Scholar
Li, F.
(2020) Mode comparability study for the TOEFL Primary tests (Unpublished technical report). ETS.Google Scholar
Monfils, L, & Manna, V. F.
this volume). Considerations in developing vertical scales for language tests. In S. Papageorgiou & V. F. Manna Eds. Meaningful language test scores: Research to enhance score interpretation John Benjamins
Papageorgiou, S., & Baron, P. A.
(2017) Using the Common European Framework of Reference for young learners’ English language proficiency assessments. In M. K. Wolf & Y. G. Butler (Eds.), English language proficiency assessments for young learners (pp. 136–152). Routledge. DOI logoGoogle Scholar
Papageorgiou, S., Davis, L., Norris, J. M., Garcia Gomez, P., Manna, V. F., & Monfils, L.
(2021) Design framework for the TOEFL® Essentials™ test 2021 (Research Memorandum No. RM–21–03). ETS. Retrieved on 7 February 2023 from [URL]Google Scholar
Papageorgiou, S., Morgan, R., & Becker, V.
(2015) Enhancing the interpretability of the overall results of an international test of English language proficiency. International Journal of Testing, 15(4), 310–336. DOI logoGoogle Scholar
Papageorgiou, S., Tannenbaum, R. J., Bridgeman, B., & Cho, Y.
(2015) The association between TOEFL iBT® test scores and the Common European Framework of Reference (CEFR) levels (Research Memorandum No. RM–15–06). ETS. Retrieved on 7 February 2023 from [URL]Google Scholar
Papageorgiou, S., Wu, S., Hsieh, C.-N., Tannenbaum, R. J., & Cheng, M. M.
(2019) Mapping the TOEFL iBT® test scores to China’s Standards of English Language Ability: Implications for score interpretation and use (Research Report No. TOEFL-RR–89). ETS. DOI logoGoogle Scholar
Powers, D., Schedl, M., & Papageorgiou, S.
(2017) Facilitating the interpretation of English language proficiency scores: Combining scale anchoring and test score mapping methodologies. Language Testing, 34(2), 175–195. DOI logoGoogle Scholar
So, Y., Wolf, M. K., Hauck, M. C., Mollaun, P., Rybinski, P., Tumposky, D., & Wang, L.
(2015) TOEFL Junior® Design Framework (TOEFL Junior® Research Report TOEFL JR–02). ETS. DOI logoGoogle Scholar
Tannenbaum, R. J., & Baron, P. A.
(2011) Mapping TOEFL® ITP scores onto the Common European Framework of Reference. (ETS Research Memorandum RM–11–33). ETS. Retrieved on 7 February 2023 from [URL]Google Scholar
Wang, L., & Papageorgiou, S.
this volume). Scale anchoring methodology for developing revised performance level descriptors for the TOEFL iBT test. In S. Papageorgiou & V. F. Manna Eds. Meaningful language test scores: Research to enhance score interpretation John Benjamins