Cover not available
Part of
Psycholinguistic and Cognitive Inquiries into Translation and Interpreting
Edited by Aline Ferreira and John W. Schwieter
[Benjamins Translation Library 115] 2015
► pp. 145–174

Post-editing machine translation

A usability test for professional translation settings

ORCID logo | Copenhagen Business School
| Johannes Gutenberg-Universität Mainz
| Johannes Gutenberg-Universität Mainz

Traditionally, the quality of machine translation (MT) output at best was sufficient to serve as an informative translation for users without any knowledge of the source language but not for the purpose of professional translation. However, still restricted to limited scenarios dependent on language pair and text type, MT quality has improved in such a way that it has found its way into professional translation workflows–especially when software localization and technical documentation are concerned. With this development in mind, the research questions of our study focus on the empirical investigation of the efficiency of post-editing and on typical revision strategies and processes. We present an empirical comparison of three translation tasks using Translog-II and Tobii eyetracking, in which 24 translators translate 6 English texts into German: two of the texts were translated from scratch, two other texts were pre-translated with Google MT, which the translator then had to post-edit and in a third task, two Google pre-translated texts had to be post-edited without the translator being able to consult the source text. We use keylogging, eyetracking and retrospective interviews to track back the different (un)conscious cognitive processes and problems involved in the different tasks. On the basis of this multi-method approach, we compare post-editing strategies to translation strategies. Furthermore, processing time as well as cognitive efforts during translations are contrasted and discussed.

Published online: 22 January 2015
DOI logo
https://doi.org/10.1075/btl.115.07car
References
Adriaens, Geert, Roger Havenith, Rick Wocjik, and Bruno Tersago
1996  First International Workshop on Controlled Language Applications. CLAW 1996 , Leuven, Belgium.
Adriaens, Geert, Jeffrey Allen, Arendse Bernth, Kurt Godden, Teruko Mitamura, Eric Nyberg, Rick Wocjik, and Rémi Zajac
2000  Third International Workshop on Controlled Language Applications. CLAW 2000 , Seattle, Washington.
Aymerich, Julia
2005 Using Machine Translation for Fast, Inexpensive, and Accurate Health Information Assimilation and Dissemination: Experiences at the Pan American Health Organization. Salvador – Bahia, Brazil: 9th World Congress on Health Information and Libraries. Accessed from: [URL], March 13, 2013.Google Scholar
Aziz, Wilker, Sheila C.M. Sousa, and Lucia Specia
2012 “PET: A Tool for Post-editing and Assessing Machine Translation.” Presented at the Eighth International Conference on Language Resources and Evaluation , Istanbul, Turkey.
Carl, Michael
2012a “Translog-II: A Program for Recording User Activity Data for Empirical Translation Process Research.” Presented at the Eighth International Conference on Language Resources and Evaluation , Istanbul, Turkey.
2012b “The CRITT TPR-DB 1.0: A Database for Empirical Human Translation Process Research.” In Proceedings of the AMTA 2012 Workshop on Post-Editing Technology and Practice (WPTP 2012), ed. by Sharon O’Brien, Michel Simard, and Lucia Specia, 9–18. Stroudsburg, PA: Association for Machine Translation in the Americas (AMTA).Google Scholar
Carl, Michael, and Martin Kay
2011 “Gazing and Typing Activities during Translation: A Comparative Study of Translation Units of Professional and Student Translators.” Meta 56 (4): 952–975. DOI logoGoogle Scholar
Carl, Michael, Barbara Dragstedt, and Arnt Lykke Jakobsen
2011 “A Taxonomy of Human Translation Styles.” Accessed from: [URL], June 8, 2014.
Choudhury, Rahzeb, and Brian McConnell
2013 TAUS Translation Technology Landscape Report. Accessed from: [URL], October 29, 2013.
Doherty, Stephen, and Sharon O’Brien
2009 Can MT Output be Evaluated through Eye Tracking? MT Summit XII, Ottawa, Canada.
Doherty, Stephen, Sharon O’Brien, and Michael Carl
2010 “Eye Tracking as an MT Evaluation Technique.” Machine Translation 24 (1):1–13. DOI logoGoogle Scholar
Doherty, Stephen, and Joss Moorkens
2013 “Investigating the Experience of Translation Technology Labs: Pedagogical Implications.” Journal of Specialised Translation 19: 122–136. Accessed from: [URL], July 21, 2013.Google Scholar
Fiederer, Rebecca, and Sharon O’Brien
2009 “Quality and Machine Translation: A Realistic Objective?” Journal Of Specialised Translation 11: 52–74. Accessed from: [URL], February 3, 2013.Google Scholar
Folaron, Deborah A
2010 “Translation Tools.” In Handbook of Translation Studies: Volume 1 2010 X, ed. by Yves Gambier and Luc van Doorslaer, 429–436. Amsterdam: John Benjamins. DOI logoGoogle Scholar
He, Yifan, Yanjun Ma, Johann Roturier, Andy Way, and Josef van Genabith
2010 “Improving the Post-editing Experience using Translation Recommendation: A user Study.” Proceedings of the 9th Annual AMTA Conference , 247–256, Denver. Accessed from: [URL], June 8, 2014.
Hofmann, Sascha
2012 Prozessgestütztes Übersetzen: vom funktionsorientierten Übersetzungsprozess zum Geschäftsprozessmodell für die Dienstleistung Übersetzen. Lichtenberg: Harland media.Google Scholar
Hvelplund, Kristian Tangsgaard
2011 Allocation of Cognitive Resources in Translation an Eye-tracking and Key-logging Study. PhD thesis, Department of International Language Studies and Computational Linguistics, Copenhagen Business School.
Jakobsen, Arnt Lykke
1999 “Translog Documentation.” Copenhagen Studies in Language 24: 149–184.Google Scholar
2011 “Tracking Translators’ Keystrokes and Eye Movements with Translog.” In Methods and Strategies of Process Research Kapitel 4 (Benjamins Translation Library, Vol. 94), ed. by C. Alvstad, A. Hild, and E. Tiselius, 37–55. Amsterdam: John Benjamins Publishing Company. DOI logoGoogle Scholar
Kirchhoff, Katrin, Anne M. Turner, Amittai Axelrod, and Francisco Saavedra
2011 “Application of Statistical Machine Translation to Public Health Information: A Feasibility Study.” Journal of the American Medical Information Association 18 (4): 473–478. Accessed from: [URL], March 13, 2013. DOI logoGoogle Scholar
Koehn, Philipp, Hieu Hoang, Alexandra Birch, Chris Callison-Burch, Marcello Federico, Nicola Bertoldi, Brooke Cowan, Wade Shen, Christine Moran, Richard Zens, Chris Dyer, Ondrej Bojar, Alexandra Constantin, and Evan Herbst
2007 “Moses: Open Source Toolkit for Statistical Machine Translation.” ACL Companion Volume. Proceedings of the Demo and Poster Sessions , 177–180. Prague, Czech Republic, June 2007. Association for Computational Linguistics.
Koehn, Philipp
2009 “A Web-Based Interactive Computer Aided Translation Tool.” Proceedings of the ACL-IJCNLP 2009 Software Demonstrations , 17–20. Suntec, Singapore.
2010a Statistical Machine Translation. Cambridge: Cambridge University Press.Google Scholar
2010b “Enabling Monolingual Translators: Post-Editing vs. Options.” Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics (HLT/NAACL) , 537–545. Los Angeles, California: Association for Computational Linguistics,. Accessed from: [URL], June 8, 2014.
Koponen, Maarit
2012 “Comparing Human Perceptions of Post-editing Effort with Post-editing Operations.” Proceedings of the 7th Workshop on Statistical Machine Translation 2012 , 181–190. Montreal, Canada, Association for Computational Linguistics. Accessed from: [URL], March 5, 2013.
Koponen, Maarit, Wilker Aziz, Luciana Ramos, and Lucia Specia
2012 “Post-editing Time as a Measure of Cognitive Effort.” Proceedings of the AMTA 2012. Workshop on Post-editing Technology and Practice (WPTP 2012) .
Krings, Hans
2001 Repairing Texts: Empirical Investigations of Machine Translation Post-editing Processes. Translated (German into English) by G. Koby, G. Shreve, K. Mischerikow, and S. Litzer; ed. by G. Koby. Kent, OH: Kent State University Press Translation Studies Series.Google Scholar
Langlais, Philippe, George Foster, and Guy Lapalme
2000 “Unit Completion for a Computer-aided Translation Typing System.” Machine Translation 15 (4): 267–294. DOI logoGoogle Scholar
Laurian, Anne-Marie
1984 “Machine Translation: What type of Post-editing on what type of Documents for what type of Users.” COLING, Association for Computational Linguistics , 236–238. Morristown, N.J. Accessed from: [URL], January 28, 2013.
Mertin, Elvira
2006 Prozessorientiertes Qualitätsmanagement im Dienstleistungsbereich Übersetzen (Leipziger Studien zur angewandten Linguistik und Translatologie, 2). Frankfurt am Main: Lang.Google Scholar
O’Brien, Sharon
2002 “Teaching Post-editing: A Proposal for Course Content.” Sixth EAMT Workshop “Teaching Machine Translation” , 99–106. UMIST, Manchester, England, November 14–15, 2002. Accessed from: [URL], February 3, 2013.
2010a “Introduction to Post‐Editing: Who, What, How and Where to Next? Tutorial.” Accessed from: [URL], February 1, 2013.
2010b “Controlled Language and Readability.” In Translation and Cognition (American Translators Association Scholarly Monograph Series XV), ed. by Shreve, Gregory, and Erik Angelone, 143–168. Amsterdam: John Benjamins. Accessed from: [URL], January 31, 2013. DOI logoGoogle Scholar
2012 “Translation as Human-Computer Interaction.” Translation Spaces 1 (1): 101–122. Accessed from: [URL], January 31, 2013. DOI logoGoogle Scholar
O’Brien, Sharon, Johann Roturier, and Giselle de Almeida
2009 “Post-Editing MT Output Views from the Researcher, Trainer, Publisher and Practitioner.” Accessed from: [URL], July 12, 2013.
Pérez, Celia Rico
2012 “A Flexible Decision Tool for Implementing Post-editing Guidelines.” Localisation Focus 11 (1): 54–65. Accessed from: [URL], June 8, 2014.Google Scholar
Plitt, Mirko, and François Masselot
2010 “A Productivity Test of Statistical Machine Translation Post-Editing in a Typical Localisation Context.” The Prague Bulletin of Mathematical Linguistics NUMBER 9 (3): 7–16.Google Scholar
Poulis, Alexandros, and David Kolovratnik
2012 “To Post-edit or not to Post-edit? Estimating the Benefits of MT Post-editing for a European Organization.” AMTA-2012: Workshop on Post-editing Technology and Practice. Proceedings , San Diego, October 28, 2012. Accessed from: [URL], June 8, 2014.
Przybocki, Mark, Greogory Sanders, and Audrey Le
2006 Edit Distance: A Metric for Machine Translation Evaluation. National Institute of Standards and Technology (NIST). Accessed from: [URL], July 22, 2013.Google Scholar
Schaeffer, Moritz, and Michael Carl
2014 “Measuring the Cognitive Effort of Literal Translation Processes.” In Workshop on Humans and Computer-assisted Translation, ed. by Ulrich Germann, 29–37. Association for Computational Linguistics, 2014.Google Scholar
Specia, Lucia
2011a Quality Estimation of Machine Translation. Abstract to Video talk on the Homepage of DCU, School of Computing, Dublin City University. Posted by gconway 07/07/2011. Accessed from: [URL], June 8, 2014.Google Scholar
2011b “Exploiting Objective Annotations for Measuring Translation Post-editing Effort.” 15th Annual Conference of the European Association for Machine Translation , 73–80, Leuven, Belgium.
2013 “Statistical Machine Translation.” In Emerging Applications of Natural Language Processing: Concepts and New Research, ed. by S. Bandyopadhyay, S.K. Naskar, and A. Ekbal, 74–109. Hershey, PA: IGI Global. DOI logoGoogle Scholar
Tatsumi, Midori, Takako Aikawa, Kentaro Yamamoto, and Hitoshi Isahara
2012 “How Good Is Crowd Post-Editing? Its Potential and Limitations.” Proceedings of the AMTA 2012 Workshop on Post-editing Technology and Practice (WPTP 2012) . Accessed from: [URL], June 8, 2014.
Temnikova, Irina
2010 “A Cognitive Evaluation Approach for a Controlled Language Post-Editing Experiment.” 7th International Conference on Language Resources and Evaluation , Valletta, Malta.
TAUS Report
2010a Postediting in Practice. Accessed from: [URL], October 29, 2013.
TAUS Translation in the 21st Century
2010b. “What machines can’t translate... yet?” Accessed from: [URL], June 8, 2014.
TAUS/CNGL: Machine Translation Post-Editing Guidelines Published. Accessed from [URL], October 29, 2013.
Van der Meer, Jaap
2013 Choose your Own Translation Future. Accessed from: [URL], January 28, 2013.
Vashee, Kirti
2011. An Exploration on Post-Editing MT – Part I. Accessed Blog from: [URL], June 8, 2014.
Wagner, Emma
1985 “Post-Editing Systran – A Challenge for Commission Translator.” Terminologie et Traduction 3.Google Scholar
1987 “Post-editing—Practical Considerations.” In The Business of Translation and Interpreting, ed. by Catriona Picken, 71–78. London: Aslib.Google Scholar
Winther-Balling, Laura, and Michael Carl
2014 “Production time across languages and tasks: A large-scale analysis using the critt translation process database.” In The Development of Translation Competence: Theories and Methodologies from Psycholinguistics and Cognitive Science, ed. by J.W. Schwieter and A. Ferreira, 239–268. Newcastle upon Tyne, UK: Cambridge Scholars Publishing.Google Scholar
Cited by

Cited by 13 other publications

Carl, Michael, Srinivas Bangalore & Moritz Schaeffer
2016. Computational linguistics and translation studies. In Border Crossings [Benjamins Translation Library, 126], ► pp. 225 ff. DOI logo
Cennamo, Ilaria & Loïc de Faria Pires
2022. Intelligence artificielle et traduction. FORUM. Revue internationale d’interprétation et de traduction / International Journal of Interpretation and Translation 20:2 ► pp. 333 ff. DOI logo
Forstner, Martin
2017.  Des Translators neue Kleider. Die Translationwirtschaft in Zeiten von Digitalisierung, Datafizierung und Big Data Management. Lebende Sprachen 62:2 ► pp. 333 ff. DOI logo
He, Shaobin, Yunhan Hao, Shijie Liu, Huidan Liu & Huadong Li
2022. Research on translation technology teaching in Chinese publications and in international English-language publications (1999-2020): a bibliometric analysis. The Interpreter and Translator Trainer 16:3 ► pp. 275 ff. DOI logo
Jiménez-Crespo, Miguel A. & Joseph V. Casillas
2021. Literal is not always easier. Translation, Cognition & Behavior 4:1 ► pp. 98 ff. DOI logo
Lesznyák, Márta, Mária Bakti & Eszter Sermann
2024. Reading takes it all? – the role of language competence and subject knowledge in legal translation. The Interpreter and Translator Trainer ► pp. 1 ff. DOI logo
Lourenço da Silva, Igor A., Fabio Alves, Marcia Schmaltz, Adriana Pagano, Derek Wong, Lidia Chao, Ana Luísa V. Leal, Paulo Quaresma, Caio Garcia & Gabriel Eduardo da Silva
2017. Chapter 4. Translation, post-editing and directionality. In Translation in Transition [Benjamins Translation Library, 133], ► pp. 108 ff. DOI logo
Malmkjær, Kirsten
2022. Translation Studies fifty years on. Translation and Translanguaging in Multilingual Contexts 8:3 ► pp. 211 ff. DOI logo
Massey, Gary & Regine Wieder
2019. Quality Assurance in Translation and Corporate Communications. In Quality Assurance and Assessment Practices in Translation and Interpreting [Advances in Linguistics and Communication Studies, ], ► pp. 57 ff. DOI logo
Mellinger, Christopher D.
2017. Translators and machine translation: knowledge and skills gaps in translator pedagogy. The Interpreter and Translator Trainer 11:4 ► pp. 280 ff. DOI logo
Moorkens, Joss, Antonio Toral, Sheila Castilho & Andy Way
2018. Translators’ perceptions of literary post-editing using statistical and neural machine translation. Translation Spaces 7:2 ► pp. 240 ff. DOI logo
Nitzke, Jean & Katharina Oster
2016. Comparing Translation and Post-editing: An Annotation Schema for Activity Units. In New Directions in Empirical Translation Process Research [New Frontiers in Translation Studies, ], ► pp. 293 ff. DOI logo
Sun, Juan, Zhi Lu, Isabel Lacruz, Lijun Ma, Lin Fan, Xiuhua Huang & Bo Zhou
2023. Chapter 4. An eye-tracking study of productivity and effort in Chinese-to-English translation and post-editing. In Translation in Transition [American Translators Association Scholarly Monograph Series, XX], ► pp. 57 ff. DOI logo

This list is based on CrossRef data as of 20 april 2024. Please note that it may not be complete. Sources presented here have been supplied by the respective publishers. Any errors therein should be reported to them.