Cover not available
Part of
Ethical Issues in Applied Linguistics Scholarship
Edited by Peter I. De Costa, Amr Rabie-Ahmed and Carlo Cinaglia
[Research Methods in Applied Linguistics 7] 2024
► pp. 235–248

Chapter 13
Ethical issues in language testing

| University of Illinois at Urbana-Champaign
| University of Illinois at Urbana-Champaign
Considerable attention has been given to ethical issues in language testing since the 1990s, with several special issues, volumes, and international conferences dedicated to fairness and justice in assessment (Davies, 1997, 2004; Kunnan, 2000; McNamara, Knoch, & Fan, 2019). In this chapter we review existing work, focusing on how ethical issues can arise at any stage of the assessment process — from initial test development through test administration, test scoring, and test use. Each of these four stages requires different quality control procedures to ensure fairness and ethical standards, namely, (1) fairness review during test development; (2) accommodations during test development; (3) bias analysis after test scoring; and (4) communication with stakeholders about ethical test use. Informed by ethical codes of practice created by language testing associations (e.g., the International Language Testing Association (ILTA) Code of Ethics), we provide specific examples to show steps that can be taken in each phase to ensure that ethical standards are upheld. Examples of ethical issues and the corresponding quality control procedures in this chapter are taken from a local, university-based English speaking test in the US. While some of the quality control procedures require language or pedagogy-related content expertise, others require specialized knowledge and skills in measurement and statistics. Finally, we conclude by recommending best practice to guide test developers and test users. We argue that assurance of ethical standards in language testing requires collaboration between language testers and other stakeholders.
Article outline
  • Introduction
  • Conceptualizing and contextualizing ethics and fairness in language testing
  • Illustrating ethics and fairness issues in language testing research and practice
    • Test development
    • Accommodations during test administration
    • Bias analysis after test scoring
    • Communication with stakeholders about ethical test use
    • Recommendations for ethical practice in language testing
  • References
This content is being prepared for publication; it may be subject to changes.
References (41)
References
American Educational Research Association, American Psychological Association, & National Council on Measurement in Education (1999). Standards for educational and psychological testing. Washington, DC.Google Scholar
Aryadoust, V. (2012). Differential item functioning in while-listening performance tests: The case of the International English Language Testing System (IELTS) listening module. International Journal of Listening, 26(1), 40–60. DOI logoGoogle Scholar
Association of Language Testers in Europe. (2001). ALTE principles of good practice. Retrieved on 26 June 2024 from [URL]
Chen, Z., & Henning, G. (1985). Linguistic and cultural bias in language proficiency tests. Language Testing, 2(2), 155–163. DOI logoGoogle Scholar
Davies, A. (Ed.). (1997). Ethics in language testing (Special issue). Language Testing, 14(3). DOI logoGoogle Scholar
(Ed.) (2004). The ethics of language assessment (Special issue). Language Assessment Quarterly, 1(2–3). DOI logoGoogle Scholar
(2010). Test fairness: A response. Language Testing, 27(2), 171–176. DOI logoGoogle Scholar
(2016). Ethical codes and unexpected consequences. In G. Fulcher & F. G. Davidson (Eds.), The Routledge handbook of language testing (pp. 455–468). Routledge.Google Scholar
Dimova, S., Yan, X., & Ginther, A. (2020). Local language testing: Design, implementation, and development. Routledge. DOI logoGoogle Scholar
Dimova, S., Yan, X., Ginther, A. (2022). Local tests, local contexts. Language Testing, 39(3). DOI logoGoogle Scholar
European Association for Language Testing and Assessment. (2006). EALTA guidelines for good practice in language testing and assessment. Retrieved on 26 June 2024 from [URL]
Fulcher, G. (2010). Practical language testing. Sage.Google Scholar
Hamp-Lyons, L. (1997). Washback, impact and validity: Ethical concerns. Language Testing, 14, 295–303. DOI logoGoogle Scholar
Ho, E. C., & Yan, X. (2021). Using community of practice to characterize collaborative essay prompt writing and its role in developing language assessment literacy for pre-service language teachers. System, 101, 102569. DOI logoGoogle Scholar
International Language Testing Association. (2000). ILTA code of ethics. Retrieved on 26 June 2024 from [URL]
Japan Language Testing Association. (2007). The JLTA code of good testing practice. Retrieved on 26 Jue 2024 from [URL]
Jia, Y. (2009). Ethical standards for language testing professionals: An introduction to five major codes. Shiken: JALT Testing & Evaluation SIG Newsletter, 13(2), 2–8. Retrieved on 26 Jue 2024 from [URL]
Kane, M. (2010). Validity and fairness. Language Testing, 27(2), 177–182. DOI logoGoogle Scholar
Koretz, D. M. (2008). Measuring up. Harvard University Press. DOI logoGoogle Scholar
Kunnan, A. (2000). Fairness and validation in language assessment: Selected papers from the 19th Language Testing Research Colloquium, Orlando, Florida. Cambridge University Press.Google Scholar
Kunnan, A. J. (2004). Test fairness. In M. Milanovic & C. Weir (Eds.), European language testing in a global context (pp. 27–48). Cambridge University Press.Google Scholar
(2008). Towards a model of test evaluation: Using the test fairness and wider context frameworks. In L. Taylor & C. Weir (Eds.), Multilingualism and assessment: Achieving transparency, assuring quality, sustaining diversity (pp. 229–251). Cambridge University Press.Google Scholar
(2010). Test fairness and Toulmin’s argument structure. Language Testing, 27(2), 183–189. DOI logoGoogle Scholar
Lynch, B. K. (1997). In search of the ethical test. Language Testing, 14, 315–327. DOI logoGoogle Scholar
McNamara, T., Knoch, U., & Fan, J. (2019). Fairness, justice, and language assessment. Oxford University Press.Google Scholar
Messick, S. (1975). The standard problem: Meaning and values in measurement and evaluation. American Psychologist, 955–966. DOI logoGoogle Scholar
(1980). Test validity and the ethics of assessment. American Psychologist, 35(11), 1012–1027. DOI logoGoogle Scholar
(1989). Validity. In R. L. Linn (Ed.), Educational measurement (3rd ed., pp. 13–103). American Council on Education and Macmillan.Google Scholar
Pill, J., & Harding, L. (2013). Defining the language assessment literacy gap: Evidence from a parliamentary inquiry. Language Testing, 30(3), 381–402. DOI logoGoogle Scholar
Popham, W. J. (2008). Classroom assessment: What teachers need to know (5th ed.). Pearson.Google Scholar
Rawls, J. (1971). A theory of justice. Harvard University Press. DOI logoGoogle Scholar
(2001). Justice as fairness: A restatement (ed. by Erin Kelly). Harvard University Press. DOI logoGoogle Scholar
Stein, Z. (2017). Social justice and educational measurement: John Rawls, the history of testing, and the future of education. Routledge.Google Scholar
Swain, M. (1984). Teaching and testing communicatively. TESL Talk, 15, 7–18.Google Scholar
Taylor, C. (1992). Multiculturalism and “the politics of recognition”. Princeton University Press.Google Scholar
Taylor, L. (2009). Developing assessment literacy. Annual Review of Applied Linguistics, 29, 21–36. DOI logoGoogle Scholar
(2013). Communicating the theory, practice and principles of language testing to test stakeholders: Some reflections. Language Testing, 30(3), 403–412. DOI logoGoogle Scholar
Walters, F. S. (2022). Ethics and fairness. In G. Fulcher & L. Harding (Eds.), The Routledge handbook of language testing (pp. 563–577). Routledge. DOI logoGoogle Scholar
Xi, X. (2010). How do we go about investigating test fairness? Language Testing, 27(2), 147–170. DOI logoGoogle Scholar
Yan, X., Cheng, L., & Ginther, A. (2019). Factor analysis for fairness: Examining the impact of task type and examinee L1 background on scores of an ITA speaking test. Language Testing, 36(2), 207–234. DOI logoGoogle Scholar
Yan, X., & Fan, J. (2021). “Am I qualified to be a language tester?” Understanding the development of language assessment literacy across three stakeholder groups. Language Testing, 38(2), 219–246. DOI logoGoogle Scholar