Article published in:

ITL - International Journal of Applied Linguistics
Vol. 168:2 (2017) ► pp. 143–163

Just testing

Applying theories of justice to high-stakes language tests

Bart Deygers | KU Leuven

Justice has been the topic of comparatively few papers in the fields of applied linguistics or language assessment. This may be due to the lack of a clear and agreed-upon definition on the one hand, or to the difficulty of operationalizing justice for test development on the other. This paper aims to remedy both problems by discussing prior conceptualizations of justice and by introducing six justice principles, which are based on theories of distributive justice that focus on human rights, fairness, equal opportunity, and dignity. The overarching aim of this paper is to advance the debate on justice, and to provide a consistent way of considering ethical and moral dilemmas that language testers face today.

Keywords: language testing, justice, language testing policy, high-stakes test, justice

Published online: 30 December 2017
https://doi.org/10.1075/itl.00001.dey

References

ALTE

(2010) Minimum standards for establishing quality profiles in ALTE examinations. ALTE. Retrieved from www.alte.org (10 March 2013).

Altman, A., & Wellman, C. H.

(2009) A liberal theory of international justice. Oxford: Oxford University Press.

Bachman, L., & Palmer, A.

(2010) Language assessment in practice. Oxford: Oxford University Pres.

Bentham, J., & Mill, J. S.

(1987) Utilitarianism and other essays. (A. Ryan, Ed.). Harmondsworth: Penguin Classics.

Boylan, M.

(2004) A just society. Lanham, MD: Rowman & Littlefield Publishers.

Connolly, T., Arkes, H. R., & Hammond, K. R.

(1999) Judgment and decision making: An interdisciplinary reader. Cambridge: Cambridge University Press.

Davies, A.

(2010) Test fairness: A response. Language Testing, 27(2), 171–176.

Deygers, B., Van den Branden, K., & Van Gorp, K.

(2017) University entrance language tests: A matter of justice. Language Testing. (Published online, awaiting print)

Dworkin, R.

(2002) Sovereign virtue: The theory and practice of equality. Cambridge, MA: Harvard University Press.

(2013a) Justice for hedgehogs. Cambridge, MA: Belknap Press.

(2013b) Taking rights seriously. London: Bloomsbury Academic.

EALTA

(2000) EALTA Guidelines for good practice in language testing and assessment. Retrieved from www.ealta.eu.org/guidelines.htm (10 March 2013).

ETS

(2016) Privacy and security. Retrieved from https://www.ets.org/legal/privacy#Collect (29 September 2016).

Foucault, M.

(1977) Discipline and punish. The birth of the prison. London: Penguin.

Fulcher, G.

(2015) Re-examining Language Testing: A Philosophical and Social Inquiry. London : Routledge.

Hobbes, T.

(1982) Leviathan. London: Penguin Classics. (Original work published 1651).

IELTS

(2016) IELTS candidate identity verification. Retrieved from http://takeielts.britishcouncil.org/book-your-test/terms-and-conditions/ielts-candidate-identity-verification (29 September 2016).

ILTA

(2000) Code of ethics. Retrieved from http://www.iltaonline.com/general/custom.asp?page=CodeofEthics (10 March 2013).

ILTA

(2007) Guidelines for practice. International Language Testing Association. Retrieved from http://www.iltaonline.com/images/pdfs/ilta_guidelines.pdf

Kane, M. T.

(2010) Validity and fairness. Language Testing, 27(2), 177–182.

(2013) Validating the interpretations and uses of test scores. Journal of Educational Measurement, 50(1), 1–73.

Kant, I.

(2016) Fundamental principles of the metaphysic of morals (T. K. Abbott, Trans.). Create Space Independent Publishing Platform. (Original work published 1785).

Kaufmann, P., Kuch, H., Neuhaeuser, C., & Webster, E.

(2010) Humiliation, degradation, dehumanization: human dignity violated. Berlin: Springer Science & Business Media.

Khan, K., & McNamara, T.

(2017) Citizenship, immigration laws, and language. In S. Canagarajah (Ed.), The Routledge handbook of migration and language (pp. 451–467). London: Routledge.

Kunnan, A. J.

(2000) Fairness and justice for all. In A. J. Kunnan (Ed.), Fairness and validation in language assessment. (pp. 1–14). Cambridge: Cambridge University Press.

(2004) Test fairness. In M. Milanovic & C. Weir (Eds.), European language testing in a global context (pp. 27–48). Cambridge: Cambridge University Press.

(2010) Test fairness and Toulmin’s argument structure. Language Testing, 27(2), 183–189.

(2014) Fairness and justice in language assessment. In A. J. Kunnan (Ed.), The companion to language assessment. New York, NY: John Wiley & Sons, Inc.

Locke, J.

(1980) Second treatise of government. (C. B. Macpherson, Ed.). Indianapolis, IN: Hackett Publishing Company, Inc. (Original work published 1689).

Long, M. H.

(2005) Methodological issues in learner needs analysis. In M. H. Long (Ed.), Second language needs analysis (pp. 19–79). Cambridge: Cambridge University Press.

McNamara, T.

(2009) Australia: The dictation test redux? Language Assessment Quarterly, 6(1), 106–111.

(2012) Language assessments as shibboleths: A poststructuralist perspective. Applied Linguistics, 33(5), 564–581.

McNamara, T., & Ryan, K.

(2011) Fairness versus justice in language testing: The place of english literacy in the australian citizenship test. Language Assessment Quarterly, 8(2), 161–178.

Messick, S.

(1989) Validity. In Educational Measurement (pp. 13–103). Washington, DC: American Council on Education / Macmillan.

Mill, J. S.

(2015) On liberty, utilitarianism and other essays (M. Philp & F. Rosen, Eds.). Oxford: Oxford University Press. (Original work published 1861).

Nagel, T.

(1989) The view from nowhere. Oxford: Oxford University Press.

Nelson, L. S.

(2006) Constructing policy: The unsettled question of biometric technology and privacy. In K. J. Strandburg & D. S. Raicu (Eds.), Privacy and technologies of identity (pp. 151–173). New York, NY: Springer.

Norris, J.

(2008) Validity evaluation in language assessment. New York, NY: Peter Lang.

Nozick, R.

(1974) Anarchy, state, and utopia. New York, NY: Basic Books.

Nussbaum, M.

(2002) Capabilities and social justice. International Studies Review, 4(2), 123–135.

(2004) Beyond the social contract: Capabilities and global justice. Oxford Development Studies, 32(1), 3–18.

(2006) Frontiers of justice. Disability, nationality, species membership. Cambridge, MA: Belknap Press.

(2011) Creating capabilities. The human development approach. Cambridge, MA: Belknap Press.

Phillips, D. C.

(2007) Adding complexity: Philosophical perspectives on the relationship between evidence and policy. Yearbook of the National Society for the Study of Education, 106(1), 376–402.

Pierce, C. P.

(2010) Idiot America: How stupidity became a virtue in the land of the free. New York, NY: Anchor Books.

Pogge, T.

(2010) A critique of the capability approach. In H. Brighouse & I. Robeyns (Eds.), Measuring justice: Primary goods and capabilities (pp. 17–61). Cambridge: Cambridge University Press.

Prometric

(2016) ID Management. Retrieved from https://www.prometric.com/en-us/our-solutions/test-delivery/id-management/pages/default.aspx (12 October 2016).

Rawls, J.

(1958) Justice as fairness. The Philosophical Review, 67(2), 164–194.

(1971) A Theory of Justice. Cambridge, MA: Harvard University Press.

(2001) Justice as Fairness: A Restatement. Cambridge, MA: Belknap Press.

Reisch, M.

(2014) Social justice and liberalism. In M. Reisch (Ed.), The Routledge international handbook of social justice (pp. 132–147). London: Routledge.

Rousseau, J. -J.

(2016) The social contract (G. Cole, Trans.). CreateSpace Independent Publishing Platform. (Original work published 1762).

Sandel, M. J.

(2010) Justice: What’s the right thing to do? New York, NY: Farrar, Straus and Giroux.

Saville, N.

(2010) Developing a model for investigating the impact of language assessment. Cambridge ESOL Research Notes, 42, 2–8.

Scanlon, T. M.

(1998) What we owe to each other. Cambridge, MA: Belknap Press.

Scientific American’s Board of Editors

(2014) Beware the eye spies. Scientific American, 310(1), 10–10.

Sen, A.

(1980) Equality of what? In S. M. McMurrin (Ed.), The Tanner lectures on human values (pp. 196–220). Salt Lake City, UT: University of Utah Press and Cambridge University Press.

(2010) The idea of justice. London: Penguin.

Sethi, I. K.

(2006) Biometrics: Overview and applications. In K. J. Strandburg & D. S. Raicu (Eds.), Privacy and technologies of identity (pp. 117–135). New York, NY: Springer.

Shohamy, E.

(1997) Testing methods, testing consequences: Are they ethical? Are they fair? Language Testing, 14(3), 340–349.

(2001) The power of tests. New York, NY: Pearson.

Strik, T., Böcker, A., Luiten, M., & van Oers, R.

(2010) The INTEC project: Synthesis report. Nijmegen: Radboud University.

Tistarelli, M., Li, S. Z., & Chellappa, R.

(Eds.) (2009) Ethics and policy of biometrics. In Handbook of remote biometrics for surveillance and security (pp. 293–312). New York, NY: Springer.

Toulmin, S.

(2001) Return to reason. Cambridge, MA: Harvard University Press.

(2003) The uses of argument (Updated edition). Cambridge: Cambridge University Press.

UN General Assembly

(1948, December 10). Universal declaration of human rights. UN General Assembly. Retrieved from http://www.ohchr.org/EN/UDHR/Documents/UDHR_Translations/eng.pdf (19 September 2016)

Valentini, L.

(2009) Coercion and (global) justice: Towards a unified framework. CSSJ Working Papers Series, SJ010, 1–23.

(2012) Justice in a globalized world: A normative framework. Oxford: Oxford University Press.

Van Avermaet, P.

(2009) Fortress Europe? Language policy regimes for immigration and citizenship. In G. Hogan-Brun, C. Mar-Molinero, & P. Stevenson (Eds.), Discourses on language and integration (pp. 15–44). Amsterdam: John Benjamins.

Van Oers, R.

(2013) Citizenship tests in Germany, the Netherlands and the UK. Unpublished PhD dissertation. Radboud University Nijmegen.

Walters, S. F.

(2012) Fairness. In G. Fulcher & F. Davidson (Eds.), The Routledge handbook of language testing (pp. 469–479). London: Routledge.

Walzer, M.

(1983) Spheres of justice. New York, NY: Basic Books.

Weir, C.

(2005) Language testing and validation. New York, NY: Palgrave Macmillan.

Witkin, S. L., & Irving, A.

(2014) Postmodern perspectives on social justice. In M. Reisch (Ed.), The Routledge international handbook of social justice (pp. 188–202). London: Routledge.

Cited by

Cited by other publications

Annick De Houwer & Lourdes Ortega

2018. In The Cambridge Handbook of Bilingualism,

Deygers, Bart

2019. In Second Handbook of English Language Teaching [Springer International Handbooks of Education, ], ► pp. 541 ff.

Deygers, Bart

2019. In Second Handbook of Information Technology in Primary and Secondary Education [Springer International Handbooks of Education, ], ► pp. 1 ff.

Hamid, M. Obaidul, Ian Hardy & Vicente Reyes

2019. Test-takers’ perspectives on a global test of English: questions of fairness, justice and validity. Language Testing in Asia 9:1

Hamid, M. Obaidul, Ngoc T. H. Hoang & Andy Kirkpatrick

2019. Language tests, linguistic gatekeeping and global mobility. Current Issues in Language Planning 20:3 ► pp. 226 ff.

This list is based on CrossRef data as of 01 december 2019. Please note that it may not be complete. Sources presented here have been supplied by the respective publishers. Any errors therein should be reported to them.