Cover not available
Part of
Meaningful Language Test Scores: Research to enhance score interpretation
Edited by Spiros Papageorgiou and Venessa F. Manna
[Innovations in Language Learning and Assessment 1] 2023
► pp. 158–167

Chapter 9
Literacy, transparency, and (mis)interpretations in communicating with testing stakeholders

| University of Ottawa

In this chapter, I reflect upon the common goal of the contributors of this volume – as evident in its title – to enhance score meaning for test users. I first discuss this goal related to concepts such as language assessment literacy and transparency. Then I focus on the score interpretation process itself. The contributions of this volume arguably represent the first half of this process: the provision of research outcomes and other information designed to make scores more meaningful. The second half of the process begins with communication of these research outcomes to the intended audience. I suggest a program of research for this second half of the interpretation process to examine whether this research information is taken up and understood as intended.

Article outline
  • Introduction
  • Transparency as a motivation to provide enhanced information to score users
  • Critical moments of (mis)interpretation
    • Possible misinterpretations related to the common vertical scale
    • Possible misinterpretations related to scale anchoring
    • Possible misinterpretations in score mapping
  • Research program
    • Phase 1: Identifying misconceptions
    • Phase 2: Remediating misconceptions (and evaluating success)
  • Conclusion
  • References
Published online: 29 June 2023
DOI logo
https://doi.org/10.1075/illa.1.09bak
References
Bloor, M., Frankland, J., Thomas, M., & Robson, K.
(2001). Focus groups in social research.  Sage. DOI logoGoogle Scholar
Chi, M. T. H.
(2008) Three types of conceptual change: Belief revision, mental model transformation, and categorical shift. In S. Vosniadou (Ed.), Handbook of research on conceptual change (pp. 61–82). Lawrence Erlbaum Associates.Google Scholar
(2013) Two kinds and four sub-types of misconceived knowledge, ways to change it and the learning outcomes. In S. Vosniadou (Ed.), International handbook of research on conceptual change (pp. 49–70). Routledge. DOI logoGoogle Scholar
Davis, L., Garcia Gomez, P., Li, S., & Manna, V. F.
this volume). Mapping TOEFL Essentials speaking and writing scores to the CEFR levels. In S. Papageorgiou & V. F. Manna Eds. Meaningful language test scores: Research to enhance score interpretation John Benjamins
Gu, L., Li, Y., Monfils, L., & Papageorgiou, S.
this volume). Statistical methodology for developing vertical scales for language tests. In S. Papageorgiou & V. F. Manna Eds. Meaningful language test scores: Research to enhance score interpretation John Benjamins
Jankowski, N., & Provezis, S.
(2014) Neoliberal ideologies, governmentality and the academy: An examination of accountability through assessment and transparency. Educational Philosophy and Theory, 46(5), 475–487. DOI logoGoogle Scholar
Jönsson, A.
(2014) Rubrics as a way of providing transparency in assessment. Assessment and Evaluation in Higher Education, 39(7), 840–852. DOI logoGoogle Scholar
Jönsson, A., & Prins, F.
(2019) Editorial: Transparency in assessment: Exploring the influence of explicit assessment criteria. Frontiers in Education, 3, 1–3. DOI logoGoogle Scholar
Kolen
(2011) Issues associated with vertical scales for PARCC Assessments. Retrieved on 8 February 2023 from [URL]
Krueger, R. A., & Casey, M. A.
(2014). Focus groups: A practical guide for applied research. Sage.Google Scholar
Li, T., & Papageorgiou, S.
this volume). Designing and executing a score concordance study to establish score mapping to proficiency levels. In S. Papageorgiou & V. F. Manna Eds. Meaningful language test scores: Research to enhance score interpretation John Benjamins
Noddings, N.
(2012) The caring relation in teaching. Oxford Review of Education, 38(6), 771–781. DOI logoGoogle Scholar
Papageorgiou, S., & Manna, V. F.
this volume). Overview of research to enhance score interpretation. In S. Papageorgiou & V. F. Manna Eds. Meaningful language test scores: Research to enhance score interpretation John Benjamins
Wang, L., & Papageorgiou, S.
this volume). Scale anchoring methodology for developing revised performance level descriptors for the TOEFL iBT test. In S. Papageorgiou & V. F. Manna Eds. Meaningful language test scores: Research to enhance score interpretation John Benjamins
Weideman, A.
(2006) Transparency and accountability in applied linguistics. Southern African Linguistics and Applied Language Studies, 24(1), 71–86. DOI logoGoogle Scholar