Chapter published in:Translation in Transition: Between cognition, computing and technology
Edited by Arnt Lykke Jakobsen and Bartolomé Mesa-Lao
[Benjamins Translation Library 133] 2017
► pp. 162–186
From process to product
Links between post-editing effort and post-edited quality
Post-editing of machine translation (MT) is now increasingly implemented in the human translation workflow after studies in both industry and academia have demonstrated the efficacy of this practice. Post-editing still involves open questions, however, such as how best to train post-editors and how to estimate the effort required by post-editing tasks. In attempting to address some of these questions, many previous studies investigate the post-editing process, but less research has focused on the post-edited product. This chapter examines the link between the process and product of post-editing by checking to see how post-editing effort data relates to the quality of post-edited texts, assessed in terms of fluency (linguistic quality) and adequacy (translation accuracy). A statistical analysis indicated that the association between editing operations and the fluency of post-edited texts is dependent on the quality of the raw MT output. Interestingly, a negative association was observed between the number of eye fixations on the text and the quality of the post-edited translations. The chapter shows empirical evidence supporting the distinction between the concepts of translation fluency and adequacy, and postulates that automatic processes play a central role in post-editing performance.
Published online: 30 September 2017
Aziz, Wilker, Sheila Castilho, and Lucia Specia
2012 “PET: a Tool for Post-Editing and Assessing Machine Translation.” In Proceedings of the Eighth International Conference on Language Resources and Evaluation (LREC 2012), 21–27 May 2012, Istanbul, Turkey. ed. by Nicoletta Calzolari, Khalid Choukri, Thierry Declerck, Mehmet Uğur Doğan, Bente Maegaard, Joseph Mariani, Jan Odijk, and Stelios Piperidis, 3982–3987.
Baayen, R. Harald
Callison-Burch, Chris, Cameron Fordyce, Philipp Koehn, Christof Monz, and Josh Schroeder
Carl, Michael, Barbara Dragsted, Jakob Elming, Daniel Hardt, and Arnt L. Jakobsen
Christensen, Rune Haubo B.
Daems, Joke, Lieve Macken, and Sonia Vandepitte
2013 “Quality as the Sum of its Parts: a Two-Step Approach for the Identification of Translation Problems and Translation Quality Assessment for HT and MT+PE.” In Proceedings of the MT Summit XIV Workshop on Post-Editing Technology and Practice (WPTP2), 2–6 September 2013, Nice, France, ed. by Sharon O’Brien, Michel Simard, and Lucia Specia, 63–71.
de Almeida, Giselle
2013 Translating the Post-Editor: an Investigation of Post-Editing Changes and Correlations with Professional Experience across Two Romance Languages . PhD Thesis, Dublin City University.
de Groot, Annette M.B.
Denkowski, Michael, and Alon Lavie
2011 “Meteor 1.3: Automatic Metric for Reliable Optimization and Evaluation of Machine Translation Systems.” In Proceedings of the Sixth Workshop on Statistical Machine Translation (EMNLP 2011), 30–31 July 2011, Edinburgh, UK, 85–91.
Depraetere, Ilse, Nathalie De Sutter, and Arda Tezcan
2014 “Post-Edited Quality, Post-Editing Behaviour and Human Evaluation: a Case Study.” In Post-Editing of Machine Translation: Processes and Applications, ed. by Sharon O'Brien, Laura Winther Balling, Michael Carl, Michel Simard, and Lucia Specia, 78–108. Newcastle upon Tyne: Cambridge Scholars Publishing.
Green, Spence, Jason Chuang, Jeffrey Heer, and Christopher D. Manning
2014 “Predictive Translation Memory: a Mixed-Initiative System for Human Language Translation.” In Proceedings of the 27th Annual ACM Symposium on User Interface Software and Technology , 05–08 October 2014, Honolulu, USA, 177–187.
Green, Spence, Jeffrey Heer, and Christopher D. Manning
2013 “The Efficacy of Human Post-Editing for Language Translation.” In ACM Human Factors in Computing Systems (CHI), 27 April–2 May 2013, Paris, France, 439–448.
Gopher, Daniel, and Emanuel Donchin
2014 “The Role of Professional Experience in Post-Editing from a Quality and Productivity Perspective.” In Post-Editing of Machine Translation: Processes and Applications, ed. by Sharon O'Brien, Laura Winther Balling, Michael Carl, Michel Simard and Lucia Specia, 51–76. Newcastle upon Tyne: Cambridge Scholars Publishing.
2007 “Manual MT Post-Editing: If It’s Not Broken, Don’t Fix It!” Translation Journal 11 (4). Accessed 26 June 2014. http://www.translationjournal.net/journal/42mt.htm.
Hogarth, Robin M.
Hvelplund, Kristian Tangsgaard
2011 Allocation of Cognitive Resources in Translation: An Eye-Tracking and Key-Logging Study . PhD Thesis, Copenhagen Business School.
Jensen, K.T. Hvelplund, Annette Sjørup, and Laura Winther Balling
2012 “Comparing Human Perceptions of Post-Editing Effort with Post-Editing Operations.” In Proceedings of the Seventh Workshop on Statistical Machine Translation (NAACL 2012), 7–8 June 2012, Montreal, Canada, 181–190.
Koponen, Maarit, Wilker Aziz, Luciana Ramos, and Lucia Specia
2012 “Post-Editing Time as a Measure of Cognitive Effort.” In Proceedings of the AMTA 2012 Workshop on Post-Editing Technology and Practice (WPTP 2012) , 28 October 2012, San Diego, USA, ed. by Sharon O'Brien, Michel Simard, and Lucia Specia, 10.
Krings, Hans P.
Lin, Chin-Yew, and Franz Josef Och
2004 “ORANGE: a Method for Evaluating Automatic Evaluation Metrics for Machine Translation.” In Proceedings of the 20th International Conference on Computational Linguistics , 23–27 August 2004, Geneva, Switzerland, 7.
Meara, Paul, and Barbara Buxton
2015 Community Post-Editing of Machine-Translated User-Generated Content . PhD Thesis, Dublin City University.
Norman, Donald A., and Tim Shallice
Paas, Fred G.
Plitt, Mirko, and François Masselot
Snijders, Tom A.B., and Roel Bosker
Snover, Matthew, Bonnie Dorr, Richard Schwartz, Linnea Micciulla, and John Makhoul
2006 “A Study of Translation Edit Rate with Targeted Human Annotation.” In Proceedings of the 7th Conference of the Association for Machine Translation of the Americas – Visions for the Future of Machine Translation , 8–12 August 2006, Cambridge, USA, 223–231.
2013 Adequacy/Fluency Guidelines. Accessed 17 June 2015. https://www.taus.net/think-tank/best-practices/evaluate-best-practices/adequacy-fluency-guidelines.
2010 MT Post-Editing Guidelines. Accessed 17 June 2015. https://www.taus.net/think-tank/best-practices/postedit-best-practices/machine-translation-post-editing-guidelines.
2010 “A Cognitive Evaluation Approach for a Controlled Language Post-Editing Experiment.” In Proceedings of the 7th International Conference on Language Resources and Evaluation (LREC 2010) , 19–21 May 2010, Valetta, Malta, 3485–3490.
2012 Determining the Tobii I-VT Fixation Filter’s Default Values: Method Description and Results Discussion. Accessed 20 December 2013. http://www.tobii.com/Global/Analysis/Training/WhitePapers/Tobii_WhitePaper_DeterminingtheTobiiI-VTFixationFilter’sDefaultValues.pdf.
Unsworth, Nash, Richard P. Heitz, Josef C. Schrock, and Randall W. Engle
Vieira, Lucas N.
2016 Cognitive Effort in Post-Editing of Machine Translation: Evidence from Eye Movements, Subjective Ratings, and Think-Aloud Protocols . PhD Thesis, Newcastle University.
Cited by 4 other publications
Albl-Mikasa, Michaela, Maureen Ehrensberger-Dow, Andrea Hunziker Heeb, Caroline Lehr, Michael Boos, Matthias Kobi, Lutz Jäncke & Stefan Elmer
de Faria Pires, Loïc
Koponen, Maarit, Leena Salmi & Markku Nikulin
This list is based on CrossRef data as of 23 november 2021. Please note that it may not be complete. Sources presented here have been supplied by the respective publishers. Any errors therein should be reported to them.