Article published in:Psycholinguistic and Cognitive Inquiries into Translation and Interpreting
Edited by Aline Ferreira and John W. Schwieter
[Benjamins Translation Library 115] 2015
► pp. 145–174
Post-editing machine translation
A usability test for professional translation settings
Traditionally, the quality of machine translation (MT) output at best was sufficient to serve as an informative translation for users without any knowledge of the source language but not for the purpose of professional translation. However, still restricted to limited scenarios dependent on language pair and text type, MT quality has improved in such a way that it has found its way into professional translation workflows–especially when software localization and technical documentation are concerned. With this development in mind, the research questions of our study focus on the empirical investigation of the efficiency of post-editing and on typical revision strategies and processes. We present an empirical comparison of three translation tasks using Translog-II and Tobii eyetracking, in which 24 translators translate 6 English texts into German: two of the texts were translated from scratch, two other texts were pre-translated with Google MT, which the translator then had to post-edit and in a third task, two Google pre-translated texts had to be post-edited without the translator being able to consult the source text. We use keylogging, eyetracking and retrospective interviews to track back the different (un)conscious cognitive processes and problems involved in the different tasks. On the basis of this multi-method approach, we compare post-editing strategies to translation strategies. Furthermore, processing time as well as cognitive efforts during translations are contrasted and discussed.
Published online: 22 January 2015
Cited by 7 other publications
Carl, Michael, Srinivas Bangalore & Moritz Schaeffer
Jiménez-Crespo, Miguel A. & Joseph V. Casillas
Lourenço da Silva, Igor A., Fabio Alves, Marcia Schmaltz, Adriana Pagano, Derek Wong, Lidia Chao, Ana Luísa V. Leal, Paulo Quaresma, Caio Garcia & Gabriel Eduardo da Silva
Massey, Gary & Regine Wieder
Mellinger, Christopher D.
Moorkens, Joss, Antonio Toral, Sheila Castilho & Andy Way
This list is based on CrossRef data as of 10 september 2021. Please note that it may not be complete. Sources presented here have been supplied by the respective publishers. Any errors therein should be reported to them.
Adriaens, Geert, Roger Havenith, Rick Wocjik, and Bruno Tersago
1996 First International Workshop on Controlled Language Applications. CLAW 1996 , Leuven, Belgium.
Adriaens, Geert, Jeffrey Allen, Arendse Bernth, Kurt Godden, Teruko Mitamura, Eric Nyberg, Rick Wocjik, and Rémi Zajac
2000 Third International Workshop on Controlled Language Applications. CLAW 2000 , Seattle, Washington.
2005 Using Machine Translation for Fast, Inexpensive, and Accurate Health Information Assimilation and Dissemination: Experiences at the Pan American Health Organization. Salvador – Bahia, Brazil: 9th World Congress on Health Information and Libraries. Accessed from: http://www.paho.org/english/am/gsp/tr/2005_ICML9-Aymerich.pdf, March 13, 2013.
Aziz, Wilker, Sheila C.M. Sousa, and Lucia Specia
2012 “PET: A Tool for Post-editing and Assessing Machine Translation.” Presented at the Eighth International Conference on Language Resources and Evaluation , Istanbul, Turkey.
2012a “Translog-II: A Program for Recording User Activity Data for Empirical Translation Process Research.” Presented at the Eighth International Conference on Language Resources and Evaluation , Istanbul, Turkey.
2012b “The CRITT TPR-DB 1.0: A Database for Empirical Human Translation Process Research.” In Proceedings of the AMTA 2012 Workshop on Post-Editing Technology and Practice (WPTP 2012), ed. by Sharon O’Brien, Michel Simard, and Lucia Specia, 9–18. Stroudsburg, PA: Association for Machine Translation in the Americas (AMTA).
Carl, Michael, and Martin Kay
Carl, Michael, Barbara Dragstedt, and Arnt Lykke Jakobsen
2011 “A Taxonomy of Human Translation Styles.” Accessed from: http://translationdirectory.com/articles/article2321.php, June 8, 2014.
Choudhury, Rahzeb, and Brian McConnell
2013 TAUS Translation Technology Landscape Report. Accessed from: http://www.translationautomation.com/reports/taus-translation-technology-landscape-report, October 29, 2013.
Doherty, Stephen, and Sharon O’Brien
2009 Can MT Output be Evaluated through Eye Tracking? MT Summit XII, Ottawa, Canada.
Doherty, Stephen, Sharon O’Brien, and Michael Carl
Doherty, Stephen, and Joss Moorkens
2013 “Investigating the Experience of Translation Technology Labs: Pedagogical Implications.” Journal of Specialised Translation 19: 122–136. Accessed from: http://www.jostrans.org/issue19/art_doherty.php, July 21, 2013.
Fiederer, Rebecca, and Sharon O’Brien
2009 “Quality and Machine Translation: A Realistic Objective?” Journal Of Specialised Translation 11: 52–74. Accessed from: http://www.jostrans.org/issue11/art_fiederer_obrien.pdf, February 3, 2013.
Folaron, Deborah A.
He, Yifan, Yanjun Ma, Johann Roturier, Andy Way, and Josef van Genabith
2010 “Improving the Post-editing Experience using Translation Recommendation: A user Study.” Proceedings of the 9th Annual AMTA Conference , 247–256, Denver. Accessed from: http://doras.dcu.ie/15803/, June 8, 2014.
Hvelplund, Kristian Tangsgaard
2011 Allocation of Cognitive Resources in Translation an Eye-tracking and Key-logging Study. PhD thesis, Department of International Language Studies and Computational Linguistics, Copenhagen Business School.
Kirchhoff, Katrin, Anne M. Turner, Amittai Axelrod, and Francisco Saavedra
2011 “Application of Statistical Machine Translation to Public Health Information: A Feasibility Study.” Journal of the American Medical Information Association 18 (4): 473–478. Accessed from: http://jamia.bmj.com/content/18/4/473.full.pdf+html, March 13, 2013.
Koehn, Philipp, Hieu Hoang, Alexandra Birch, Chris Callison-Burch, Marcello Federico, Nicola Bertoldi, Brooke Cowan, Wade Shen, Christine Moran, Richard Zens, Chris Dyer, Ondrej Bojar, Alexandra Constantin, and Evan Herbst
2007 “Moses: Open Source Toolkit for Statistical Machine Translation.” ACL Companion Volume. Proceedings of the Demo and Poster Sessions , 177–180. Prague, Czech Republic, June 2007. Association for Computational Linguistics.
2009 “A Web-Based Interactive Computer Aided Translation Tool.” Proceedings of the ACL-IJCNLP 2009 Software Demonstrations , 17–20. Suntec, Singapore.
2010b “Enabling Monolingual Translators: Post-Editing vs. Options.” Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics (HLT/NAACL) , 537–545. Los Angeles, California: Association for Computational Linguistics,. Accessed from: http://www.aclweb.org/anthology/N/N10/N10-1078.pdf, June 8, 2014.
2012 “Comparing Human Perceptions of Post-editing Effort with Post-editing Operations.” Proceedings of the 7th Workshop on Statistical Machine Translation 2012 , 181–190. Montreal, Canada, Association for Computational Linguistics. Accessed from: http://www.statmt.org/wmt12/pdf/WMT23.pdf, March 5, 2013.
Koponen, Maarit, Wilker Aziz, Luciana Ramos, and Lucia Specia
2012 “Post-editing Time as a Measure of Cognitive Effort.” Proceedings of the AMTA 2012. Workshop on Post-editing Technology and Practice (WPTP 2012) .
Langlais, Philippe, George Foster, and Guy Lapalme
1984 “Machine Translation: What type of Post-editing on what type of Documents for what type of Users.” COLING, Association for Computational Linguistics , 236–238. Morristown, N.J. Accessed from: http://acl.ldc.upenn.edu/P/P84/P84-1051.pdf, January 28, 2013.
2002 “Teaching Post-editing: A Proposal for Course Content.” Sixth EAMT Workshop “Teaching Machine Translation” , 99–106. UMIST, Manchester, England, November 14–15, 2002. Accessed from: http://www.mt-archive.info/EAMT-2002-OBrien.pdf, February 3, 2013.
2010a “Introduction to Post‐Editing: Who, What, How and Where to Next? Tutorial.” Accessed from: http://amta2010.amtaweb.org/AMTA/papers/6-01-ObrienPostEdit.pdf, February 1, 2013.
2010b “Controlled Language and Readability.” In Translation and Cognition (American Translators Association Scholarly Monograph Series XV), ed. by Shreve, Gregory, and Erik Angelone, 143–168. Amsterdam: John Benjamins. Accessed from: http://doras.dcu.ie/17153/, January 31, 2013.
2012 “Translation as Human-Computer Interaction.” Translation Spaces 1 (1): 101–122. Accessed from: http://doras.dcu.ie/17541/1/Translation_as_HCI_OBrien.pdf, January 31, 2013.
O’Brien, Sharon, Johann Roturier, and Giselle de Almeida
2009 “Post-Editing MT Output Views from the Researcher, Trainer, Publisher and Practitioner.” Accessed from: http://www.mt-archive.info/MTS-2009-O'Brien-ppt.pdf, July 12, 2013.
Pérez, Celia Rico
2012 “A Flexible Decision Tool for Implementing Post-editing Guidelines.” Localisation Focus 11 (1): 54–65. Accessed from: http://www.localisation.ie/resources/locfocus/LocalisationFocusVol11_1Web.pdf, June 8, 2014.
Plitt, Mirko, and François Masselot
Poulis, Alexandros, and David Kolovratnik
2012 “To Post-edit or not to Post-edit? Estimating the Benefits of MT Post-editing for a European Organization.” AMTA-2012: Workshop on Post-editing Technology and Practice. Proceedings , San Diego, October 28, 2012. Accessed from: http://www.mt-archive.info/AMTA-2012-Poulis.pdf, June 8, 2014.
Przybocki, Mark, Greogory Sanders, and Audrey Le
2006 Edit Distance: A Metric for Machine Translation Evaluation. National Institute of Standards and Technology (NIST). Accessed from: http://www.mt-archive.info/LREC-2006-Przybocki.pdf, July 22, 2013.
Schaeffer, Moritz, and Michael Carl
2011a Quality Estimation of Machine Translation. Abstract to Video talk on the Homepage of DCU, School of Computing, Dublin City University. Posted by gconway 07/07/2011. Accessed from: http://www.computing.dcu.ie/blogs/dr-lucia-specia-quality-estimation-machine-translation-4th-july-2011, June 8, 2014.
2011b “Exploiting Objective Annotations for Measuring Translation Post-editing Effort.” 15th Annual Conference of the European Association for Machine Translation , 73–80, Leuven, Belgium.
Tatsumi, Midori, Takako Aikawa, Kentaro Yamamoto, and Hitoshi Isahara
2012 “How Good Is Crowd Post-Editing? Its Potential and Limitations.” Proceedings of the AMTA 2012 Workshop on Post-editing Technology and Practice (WPTP 2012) . Accessed from: http://amta2012.amtaweb.org/AMTA2012Files/html/8/8_paper.pdf, June 8, 2014.
2010 “A Cognitive Evaluation Approach for a Controlled Language Post-Editing Experiment.” 7th International Conference on Language Resources and Evaluation , Valletta, Malta.
2010a Postediting in Practice. Accessed from: https://www.taus.net/reports/postediting-in-practice, October 29, 2013.
TAUS Translation in the 21st Century
2010b. “What machines can’t translate... yet?” Accessed from: http://www.translationautomation.com/articles/what-machines-cannot-translate-yet, June 8, 2014.
TAUS/CNGL: Machine Translation Post-Editing Guidelines Published. Accessed from http://www.cngl.ie/node/2542, October 29, 2013.
Van der Meer, Jaap
2013 Choose your Own Translation Future. Accessed from: http://langtechnews.hivefire.com/articles/share/27171, January 28, 2013.
2011. An Exploration on Post-Editing MT – Part I. Accessed Blog from: http://kv-emptypages.blogspot.de/2011/02/exploration-of-post-editing-mt-part-i.html, June 8, 2014.
Winther-Balling, Laura, and Michael Carl
2014 “Production time across languages and tasks: A large-scale analysis using the critt translation process database.” In The Development of Translation Competence: Theories and Methodologies from Psycholinguistics and Cognitive Science, ed. by J.W. Schwieter and A. Ferreira, 239–268. Newcastle upon Tyne, UK: Cambridge Scholars Publishing.