Measuring the usability of machine translation in the classroom context
Usability is a key factor for increasing adoption of machine translation. This study aims to measure the usability
of machine translation in the classroom context by comparing translation students’ machine translation post-editing output with
their manual translation in two comparable translation tasks. Three dimensions of usability were empirically measured:
efficiency, effectiveness, and satisfaction. The findings suggest that machine translation
post-editing is more efficient than human translation and post-editing produces fewer errors than human translation. While the
types of errors vary, errors in terms of accuracy outnumber those related to fluency. In addition, participants perceive the
amount of time and work that is saved when post-editing to be greater benefit than the overall utility of post-editing. Likewise,
students report a strong desire to learn post-editing skills in training programs.
Keywords: usability, machine translation, post-editing, human translation
Published online: 25 August 2020
https://doi.org/10.1075/tis.18047.yan
https://doi.org/10.1075/tis.18047.yan
References
References
Bevan, Nigel, James Carter, and Susan Harker
Bowker, Lynne
Bowker, Lynne and Jairo Buitrago Ciro
Cadwell, Patrick, Sharon O’Brien, and Carlos Teixeira
Carl, Michael
Carl, Michael and Toledo Cristina Báez
Cohen, Jacob
Daems, Joke, et al.
Daems, Joke, Sonia Vandepitte, Robert Hartsuiker, and Lieve Macken
Davis, Fred
Davis, Fred, Richard Bagozzi, and Paul Warshaw
De Almeida, Giselle and Sharon O’Brien
Doherty, Stephen and Dorothy Kenny
Doherty, Stephen and Sharon O’Brien
Ducar, Cynthia and Deborah Houk Schocket
Fiederer, Rebecca and Sharon O’Brien
Flanagan, Marian and Tina Paulsen Christensen
Germann, Ulrich
Gile, Daniel
Guerberof Arenas, Ana
Hansen, Gyde
Harrati, Nouzha, et al.
ISO9241-11
Jia, Yanfang, Michael Carl, and Xiangling Wang
Kenny, Dorothy
Kenny, Dorothy and Stephen Doherty
Kingscott, Geoffrey
Koponen, Maarit
Kortum, Philip and Frederick Oswald
Krüger, Ralph
Lacruz, Isable, Michael Denkowski, and Alon Lavie
Lewis, James R.
Lexile
Lommel, Arle, Hans Uszkoreit, and Burchardt Aljoscha
Mariana, Valerie, Troy Cox, and Alan Melby
Mellinger, Christopher D.
Mellinger, Christopher D. and Gregory M. Shreve
Mellinger, Christopher D. and Thomas A. Hanson
MetaMetrics
2018 About Lexile ® Measures for Reading. https://lexile.com/educators/understanding-lexile-measures/about-lexile-measures-for-reading. Last accessed 23 May 2020.
Moorkens, Joss
Moorkens, Joss, Antonio Toral, Sheila Castilho, and Andy Way
O’Brien, Sharon
Plitt, Mirko and François Masselot
R Core Team
2018 “R: A language and environment for statistical computing.” R Foundation for Statistical Computing. Vienna. www.R-project.org. Last accessed 23 May 2020.
Raita, Eeva and Antti Oulasvira
Rossi, Caroline
Rossi, Caroline and Jean-Pierre Chevrot
Sakamoto, Akiko
Sánchez-Gijón, Pilar and Olga Torres-Hostench
2014 “MT Post-editing into the mother tongue or into a foreign language? Spanish-to-English MT translation output post-edited by translation trainees.” In Proceedings of the Third Workshop on Post-editing Technology and Practice, ed. by Sharon O’Brien, Michel Simard and Lucia Specia, 5–17. Vancouver.
Shuttleworth, Mark
Suojanen, Tytti, Kaisa Koskinen, and Tiina Tuominen
Temizöz, Ö.
Temnikova, Irina
2010 “Cognitive evaluation approach for a controlled language PE experiment.” In Proceedings of the 7th International Conference on Language Resources and Evaluation, ed. by Nicoletta Calzolari, Khalid Choukri, Bente Maegaard, Joseph Mariani, Jan Odijk, Stelios Piperidis, Mike Rosner and Daniel Tapias, 3485–3490. Valletta.
Tirkkonen-Condit, Sonja
Trace, Jonathan, Gerriet Janssen, and Valerie Meier
Van der Heijden, Hans
Wang, Huashu
Wu, Jen-Her and Shu-Ching Wang
Yamada, Masaru
Yang, Yanxia and Xiangling Wang
Zaharias, Panagiotis
Zhai, Yuming, Aurélien Max, and Anne Vilnat