Measuring the usability of machine translation in the classroom context
Usability is a key factor for increasing adoption of machine translation. This study aims to measure the usability of machine translation in the classroom context by comparing translation students’ machine translation post-editing output with their manual translation in two comparable translation tasks. Three dimensions of usability were empirically measured: efficiency, effectiveness, and satisfaction. The findings suggest that machine translation post-editing is more efficient than human translation and post-editing produces fewer errors than human translation. While the types of errors vary, errors in terms of accuracy outnumber those related to fluency. In addition, participants perceive the amount of time and work that is saved when post-editing to be greater benefit than the overall utility of post-editing. Likewise, students report a strong desire to learn post-editing skills in training programs.
Keywords: usability, machine translation, post-editing, human translation
Published online: 25 August 2020
Bevan, Nigel, James Carter, and Susan Harker
Bowker, Lynne and Jairo Buitrago Ciro
Cadwell, Patrick, Sharon O’Brien, and Carlos Teixeira
Carl, Michael and Toledo Cristina Báez
Daems, Joke, et al.
Daems, Joke, Sonia Vandepitte, Robert Hartsuiker, and Lieve Macken
Davis, Fred, Richard Bagozzi, and Paul Warshaw
De Almeida, Giselle and Sharon O’Brien
Doherty, Stephen and Dorothy Kenny
Doherty, Stephen and Sharon O’Brien
Ducar, Cynthia and Deborah Houk Schocket
Fiederer, Rebecca and Sharon O’Brien
Flanagan, Marian and Tina Paulsen Christensen
Guerberof Arenas, Ana
Harrati, Nouzha, et al.
Jia, Yanfang, Michael Carl, and Xiangling Wang
Kenny, Dorothy and Stephen Doherty
Kortum, Philip and Frederick Oswald
Lacruz, Isable, Michael Denkowski, and Alon Lavie
Lewis, James R.
Lommel, Arle, Hans Uszkoreit, and Burchardt Aljoscha
Mariana, Valerie, Troy Cox, and Alan Melby
Mellinger, Christopher D.
Mellinger, Christopher D. and Gregory M. Shreve
Mellinger, Christopher D. and Thomas A. Hanson
2018 About Lexile ® Measures for Reading. https://lexile.com/educators/understanding-lexile-measures/about-lexile-measures-for-reading. Last accessed 23 May 2020.
Moorkens, Joss, Antonio Toral, Sheila Castilho, and Andy Way
Plitt, Mirko and François Masselot
R Core Team
2018 “R: A language and environment for statistical computing.” R Foundation for Statistical Computing. Vienna. www.R-project.org. Last accessed 23 May 2020.
Raita, Eeva and Antti Oulasvira
Rossi, Caroline and Jean-Pierre Chevrot
Sánchez-Gijón, Pilar and Olga Torres-Hostench
2014 “MT Post-editing into the mother tongue or into a foreign language? Spanish-to-English MT translation output post-edited by translation trainees.” In Proceedings of the Third Workshop on Post-editing Technology and Practice, ed. by Sharon O’Brien, Michel Simard and Lucia Specia, 5–17. Vancouver.
Suojanen, Tytti, Kaisa Koskinen, and Tiina Tuominen
2010 “Cognitive evaluation approach for a controlled language PE experiment.” In Proceedings of the 7th International Conference on Language Resources and Evaluation, ed. by Nicoletta Calzolari, Khalid Choukri, Bente Maegaard, Joseph Mariani, Jan Odijk, Stelios Piperidis, Mike Rosner and Daniel Tapias, 3485–3490. Valletta.
Trace, Jonathan, Gerriet Janssen, and Valerie Meier
Van der Heijden, Hans
Wu, Jen-Her and Shu-Ching Wang
Yang, Yanxia and Xiangling Wang
Zhai, Yuming, Aurélien Max, and Anne Vilnat