Using rating scales to assess interpretation
Practices, problems and prospects
Over the past decade, interpretation assessment has played an increasingly important role in interpreter education, professional certification, and interpreting research. The time-honored assessment method is based on analysis of (para)linguistic features of interpretation (including such items as omissions, substitutions, un/filled pauses and self-corrections). Recently, use of descriptor-based rating scales to assess interpretation has emerged as a viable alternative (e.g., Angelelli 2009; Han 2015, 2016; J. Lee 2008; Tiselius 2009), arguably providing a basis for reliable, valid and practical assessments. However, little work has been done in interpreting studies to ascertain the assumed benefits of this emerging assessment practice. Based on 17 international peer-reviewed journals over the last twelve years (2004–2015), and other related publications (e.g., scholarly books, reports, documents), this article provides an overview of practices in scale-based interpretation assessment, focusing on four major aspects: (a) rating scales; (b) raters; (c) rating procedures; (d) reporting of assessment outcomes. Problem areas and possible emerging trends in interpretation assessment are examined, identifying a number of future research needs.
Keywords: interpretation assessment, rating scales, reliability, validity
Published online: 26 April 2018
ALTA Language Services
(2007) Study of California’s court interpreter certification and registration testing. http://www.courts.ca.gov/documents/altafinalreport.pdf (accessed 10 June 2015).
American Educational Research Association, American Psychological Association, and National Council on Measurement in Education
Angelelli, C. V. & Jacobson, H. E.
Angelelli, C. V.
Arocha, I. S. & Joyce, L.
Bachman, L. F. & Palmer, A. S.[ p. 87 ]
Bachman, L. F.
Barik, H. C.
Bond, T. G. & Fox, C. M.
Bontempo, K. & Napier, J.
Bontempo, K. & B. Hutchinson
Campbell, S. & Hale, S.
Carroll, J. B.
(2011) Technical report on the development and pilot testing of the CCHI examinations. Washington, DC: Certification Commission for Healthcare Interpreters. http://www.cchicertification.org/images/pdfs/cchi%20technical%20report%20-%20public%20final.pdf (accessed 22 May 2015).
(2012) Technical report on the development and pilot testing of the Certified Healthcare Interpreter™ (CHI™) examination for Arabic and Mandarin. Washington, DC: Certification Commission for Healthcare Interpreters. http://www.cchicertification.org/images/pdfs/cchi%20arabic%20and%20mandarin%20technical%20report-final.pdf (accessed 22 May 2015).
Cheung, A. K. -F.
Choi, J. Y.
[ p. 88 ]
Dragsted, B. & Hansen, I. G.
East, M. & Young, D.
Feng, J. Z.
Fulcher, G., Davidson, F. & Kemp, J.
Goulden, N. R.
Hale, S. B. & Ozolins, U.
Hale, S., Garcia, I., Hlavac, J., Kim, M., Lai, M., Turner, B. & Slatyer, H.
(2012) Development of a conceptual overview for a new model for NAATI standards, testing and assessment. Sydney, Australia. http://www.naati.com.au/PDF/INT/INTFinalReport.pdf (accessed 22 May 2015).
Hamidi, M. & Pöchhacker, F.
Hlavac, J.[ p. 89 ]
(2014) Measuring rater variability in interpreter performance testing: Using classical test theory, G theory and Rasch measurement. Paper presented at the Biennial Conference of the Association for Language Testing and Assessment of Australia and New Zealand at the University of Queensland, 27–29 November 2014.
Han, C., & Slatyer, H.
ILR (Interagency Language Roundtable)
(2015) ILR skill level descriptions for interpretation performance. http://www.govtilr.org/Skills/interpretationSLDsapproved.htm (accessed
1 July 2015).
1 July 2015).
(2010) Diploma in Public Service Interpreting: Handbook for candidates. London, UK. http://www.iol.org.uk/qualifications/DPSI/Handbook/DPSIHB11.pdf (accessed
22 May 2015).
22 May 2015).
Iwashita, N. & Grove, E.
Jacobson, H. E.
Jamieson, J. & Poonpon, K.
(2013) Developing analytic rating guides for TOEFL iBT® integrated speaking tasks. https://www.ets.org/Media/Research/pdf/RR-13-13.pdf (accessed
12 June 2015).
12 June 2015).
Lee, S. -B.
Lee, Y. -H.
Lim, H. -O.[ p. 90 ]
Lin, I. I., Chang, F. A., & Kuo, F.
Liu, M., Chang, C. -C. & Wu, S. -C.
Llewellyn, J. P.
Lu, M., Liu, C. & Gong, X. F.
Lunz, M. E. & Stahl, J. A.
Lynch, B. K. & McNamara, T. F.
Matthews, G. & Ardemagni, E. J.
McNamara, T. F. & Adams, R. J.
(1991) Exploring rater behavior with Rasch techniques. http://files.eric.ed.gov/fulltext/ED345498.pdf (accessed 20 March 2015).
Meuleman, C. & Van Besien, F.
Moreno, M., Otero-Sabogai, R. & Newman, J.
Myford, C. M. & Wolfe, E. W.
Napier, J.[ p. 91 ]
National Center for State Courts
(2013) Federal Court Interpreter Certification Examination for Spanish/English: Examinee handbook. http://www.ncsc.org/sitecore/content/microsites/fcice/home/About-the-program/~/media/Microsites/Files/FCICE/Final%20Examinee%20Handbook%201-23-2013%20for%20online.ashx (accessed 22 May 2015).
Office of China Accreditation Tests for Translators and Interpreters
Pellatt, V., Griffiths, K. & Wu, S. -C.
Peng, K. -C.
Petronio, K. & Hale, K.
Pradas Macías, M.
PSI Services LLC
(2010) Development and validation of oral and written examinations for medical interpreter certification: Technical report. Burbank, CA. http://www.certifiedmedicalinterpreters.org//sites/default/files/oral-and-written-medical-interpreter-technical-report-final.pdf (accessed 22 May 2015).
(2013) Development and validation of oral examinations for medical interpreter certification: Mandarin, Russian, Cantonese, Korean, and Vietnamese forms. http://www.certifiedmedicalinterpreters.org/sites/default/files/tech-report-development-validation-language-forms.pdf (accessed 22 May 2015).
Ribas, M. A.
Roat, C. E.[ p. 92 ]
(2006) Certification of health care interpreters in the United States: A primer, a status report and considerations for national certification. Los Angeles, CA. http://www.calendow.org/uploadedFiles/certification_of_health_care_interpretors.pdf (accessed 22 May 2015).
Rosiers, A., Eyckmans, J. & Bauwens, D.
Rovira-Esteva, S. & Orero, P.
Russell, D. & Malcolm, K.
Sawyer, D. B.
Setton, R. & Motta, M.
Strong, M. & Rudser, S. F.
Timarová, Š. & Ungoed-Thomas, H.
Timarová, Š., Čeňková, I., Meylaerts, R., Hertog, E., Szmalec, A. & Duyck, W.
Timarová, Š. & Ungoed-Thomas, H.[ p. 93 ]
Tsagari, D. & van Deemter, R.
Turner, B., Lai, M. & Huang, N.
Upshur, J. & Turner, C. E.
Vermeiren, H., Gucht, J. V. & De Bontridder, L.
Wang, B. H.
Wang, J. -H., Napier, J., Goswell, D. & Carmichael, A.
Wang, M. W. & Stanley, J. C.
Wu, J., Liu, M. & Liao, C.
Wu, S. C.
Xi, X. -M. & Mollaun, P.
(2006) Investigating the utility of analytic scoring for the TOEFL Academic Speaking Test (TAST). https://www.ets.org/Media/Research/pdf/RR-06-07.pdf (accessed 15 June 2015).
Yan, J. X., Pan, J. & Wang, H. -H.[ p. 94 ]
Yan, J. X., Pan, J., Wu, H. & Wang, Y.
Yeh, S. -P., & Liu, M.
Zhao, N. & Dong, Y. P.
Cited by 6 other publications
Han, Chao, Sijia Chen, Rongbo Fu & Qin Fan
Han, Chao, Rui Xiao & Wei Su
Wang, Weiwei, Yi Xu, Binhua Wang & Lei Mu
이지은, Choi, Hyo-eun & You-jin Lee
This list is based on CrossRef data as of 29 august 2021. Please note that it may not be complete. Sources presented here have been supplied by the respective publishers. Any errors therein should be reported to them.