Post-editing of machine translation (MT) is now increasingly implemented in the human translation workflow after studies in both industry and academia have demonstrated the efficacy of this practice. Post-editing still involves open questions, however, such as how best to train post-editors and how to estimate the effort required by post-editing tasks. In attempting to address some of these questions, many previous studies investigate the post-editing process, but less research has focused on the post-edited product. This chapter examines the link between the process and product of post-editing by checking to see how post-editing effort data relates to the quality of post-edited texts, assessed in terms of fluency (linguistic quality) and adequacy (translation accuracy). A statistical analysis indicated that the association between editing operations and the fluency of post-edited texts is dependent on the quality of the raw MT output. Interestingly, a negative association was observed between the number of eye fixations on the text and the quality of the post-edited translations. The chapter shows empirical evidence supporting the distinction between the concepts of translation fluency and adequacy, and postulates that automatic processes play a central role in post-editing performance.
2012 “PET: a Tool for Post-Editing and Assessing Machine Translation.” In Proceedings of the
Eighth International Conference on Language Resources and Evaluation (LREC 2012), 21–27 May 2012, Istanbul, Turkey. ed. by Nicoletta Calzolari, Khalid Choukri, Thierry Declerck, Mehmet Uğur Doğan, Bente Maegaard, Joseph Mariani, Jan Odijk, and Stelios Piperidis, 3982–3987.
Baayen, R. Harald
2008Analysing Linguistic Data: a Practical Introduction to Statistics. Cambridge: Cambridge University Press.
Callison-Burch, Chris, Cameron Fordyce, Philipp Koehn, Christof Monz, and Josh Schroeder
2007 “(Meta-) Evaluation of Machine Translation.” In Proceedings of the
Second Workshop on Statistical Machine Translation
, Prague, Czech Republic, 23 June 2007, 136–158.
Carl, Michael, Barbara Dragsted, Jakob Elming, Daniel Hardt, and Arnt L. Jakobsen
2011 “The Process of Post-Editing: a Pilot Study.” In Proceedings of the 8th International NLPCS Workshop: Human-Machine Interaction in Translation, ed. by Bernardette Sharp, Michael Zock, Michael Carl, and Arnt L. Jakobsen, 131–142. Copenhagen: Samfundslitteratur.
Christensen, Rune Haubo B
2010ordinal: Regression Models for Ordinal Data. R Package Version no. 22.
Daems, Joke, Lieve Macken, and Sonia Vandepitte
2013 “Quality as the Sum of its Parts: a Two-Step Approach for the Identification of Translation Problems and Translation Quality Assessment for HT and MT+PE.” In Proceedings of the
MT Summit XIV Workshop on Post-Editing Technology and Practice (WPTP2), 2–6 September 2013, Nice, France, ed. by Sharon O’Brien, Michel Simard, and Lucia Specia, 63–71.
de Almeida, Giselle
2013Translating the Post-Editor: an Investigation of Post-Editing Changes and Correlations with Professional Experience across Two Romance Languages. PhD Thesis, Dublin City University.
2011 “Meteor 1.3: Automatic Metric for Reliable Optimization and Evaluation of Machine Translation Systems.” In Proceedings of the
Sixth Workshop on Statistical Machine Translation (EMNLP 2011), 30–31 July 2011, Edinburgh, UK, 85–91.
Depraetere, Ilse, Nathalie De Sutter, and Arda Tezcan
2014 “Post-Edited Quality, Post-Editing Behaviour and Human Evaluation: a Case Study.” In Post-Editing of Machine Translation: Processes and Applications, ed. by Sharon O'Brien, Laura Winther Balling, Michael Carl, Michel Simard, and Lucia Specia, 78–108. Newcastle upon Tyne: Cambridge Scholars Publishing.
Duchowski, Andrew T
2007Eye Tracking Methodology: Theory and Practice. London: Springer-Verlag.
Green, Spence, Jason Chuang, Jeffrey Heer, and Christopher D. Manning
2014 “Predictive Translation Memory: a Mixed-Initiative System for Human Language Translation.” In Proceedings of the
27th Annual ACM Symposium on User Interface Software and Technology
, 05–08 October 2014, Honolulu, USA, 177–187.
Green, Spence, Jeffrey Heer, and Christopher D. Manning
2013 “The Efficacy of Human Post-Editing for Language Translation.” In
ACM Human Factors in Computing Systems (CHI), 27 April–2 May 2013, Paris, France, 439–448.
Gopher, Daniel, and Emanuel Donchin
1986 “Workload: an Examination of the Concept.” In Handbook of Perception and Human Performance, Vol. 2, ed. by Kenneth R. Boff, Lloyd Kaufman, and James P. Thomas, 1–49. New York: John Wiley and Sons.
2014 “The Role of Professional Experience in Post-Editing from a Quality and Productivity Perspective.” In Post-Editing of Machine Translation: Processes and Applications, ed. by Sharon O'Brien, Laura Winther Balling, Michael Carl, Michel Simard and Lucia Specia, 51–76. Newcastle upon Tyne: Cambridge Scholars Publishing.
2007 “Manual MT Post-Editing: If It’s Not Broken, Don’t Fix It!” Translation Journal 11 (4). Accessed 26 June 2014. [URL].
Hogarth, Robin M
2005 “Deciding Analytically or Trusting Your Intuition? The Advantages and Disadvantages of Analytic and Intuitive Thought.” In The Routines of Decision Making, ed. by Tilmann Betsch, and Susanne Haberstroh, 67–82. Mahwah, NJ: Lawrence Erlbaum Associates.
Hvelplund, Kristian Tangsgaard
2011Allocation of Cognitive Resources in Translation: An Eye-Tracking and Key-Logging Study. PhD Thesis, Copenhagen Business School.
Jensen, K.T. Hvelplund, Annette Sjørup, and Laura Winther Balling
2009 “Effects of L1 Syntax on L2 Translation.” In Methodology, Technology and Innovation in Translation Process Research, ed. by Inger M. Mees, Fabio Alves, and Susanne Göpferich, 319–339. Copenhagen: Samfundslitteratur.
2012 “Comparing Human Perceptions of Post-Editing Effort with Post-Editing Operations.” In Proceedings of the
Seventh Workshop on Statistical Machine Translation (NAACL 2012), 7–8 June 2012, Montreal, Canada, 181–190.
Koponen, Maarit, Wilker Aziz, Luciana Ramos, and Lucia Specia
2012 “Post-Editing Time as a Measure of Cognitive Effort.” In Proceedings of the
AMTA 2012 Workshop on Post-Editing Technology and Practice (WPTP 2012)
, 28 October 2012, San Diego, USA, ed. by Sharon O'Brien, Michel Simard, and Lucia Specia, 10.
Krings, Hans P
2001Repairing Texts: Empirical Investigations of Machine Translation Post-Editing Processes, ed. by Geoffrey Koby. Kent, Ohio: Kent State University Press.
2005Linguistic Data Annotation Specification: Assessment of Fluency and Adequacy in Translations. Revision 1.5.
Lin, Chin-Yew, and Franz Josef Och
2004 “ORANGE: a Method for Evaluating Automatic Evaluation Metrics for Machine Translation.” In Proceedings of the
20th International Conference on Computational Linguistics
, 23–27 August 2004, Geneva, Switzerland, 7.
1996 “A Capacity Theory of Writing: Working Memory in Composition.” Educational Psychology Review 8:299–325. .
Meara, Paul, and Barbara Buxton
1987 “An Alternative to Multiple Choice Vocabulary Tests.” Language Testing 4 (2): 142–154.
2015Community Post-Editing of Machine-Translated User-Generated Content. PhD Thesis, Dublin City University.
Norman, Donald A., and Tim Shallice
1986“Attention to Action: Willed and Automatic Control of Behaviour.” In Consciousness and Self-Regulation: Advances in Research and Theory, ed. by Richard J. Davidson, Gary E. Schwarts, and David Shapiro, 1–18. New York: Plenum Press.
1999Multilevel Analysis: an Introduction to Basic and Advanced Multilevel Modelling. London: Sage.
Snover, Matthew, Bonnie Dorr, Richard Schwartz, Linnea Micciulla, and John Makhoul
2006 “A Study of Translation Edit Rate with Targeted Human Annotation.” In Proceedings of the
7th Conference of the Association for Machine Translation of the Americas – Visions for the Future of Machine Translation
, 8–12 August 2006, Cambridge, USA, 223–231.
2013Adequacy/Fluency Guidelines. Accessed 17 June 2015. [URL].
2010MT Post-Editing Guidelines. Accessed 17 June 2015. [URL].
2010 “A Cognitive Evaluation Approach for a Controlled Language Post-Editing Experiment.” In Proceedings of the
7th International Conference on Language Resources and Evaluation (LREC 2010)
, 19–21 May 2010, Valetta, Malta, 3485–3490.
2012Determining the Tobii I-VT Fixation Filter’s Default Values: Method Description and Results Discussion. Accessed 20 December 2013. [URL].
Unsworth, Nash, Richard P. Heitz, Josef C. Schrock, and Randall W. Engle
2005 “An Automated Version of the Operation Span Task.” Behavior Research Methods 37 (3): 498–505.
Vieira, Lucas N
2014 “Indices of Cognitive Effort in Machine Translation Post-Editing.” Machine Translation 28 (3–4): 187–216.
Vieira, Lucas N
2016Cognitive Effort in Post-Editing of Machine Translation: Evidence from Eye Movements, Subjective Ratings, and Think-Aloud Protocols. PhD Thesis, Newcastle University.
Cited by 4 other publications
Albl-Mikasa, Michaela, Maureen Ehrensberger-Dow, Andrea Hunziker Heeb, Caroline Lehr, Michael Boos, Matthias Kobi, Lutz Jäncke & Stefan Elmer
2019. A product and process analysis of post-editor corrections on neural, statistical and rule-based machine translation output. Machine Translation 33:1-2 ► pp. 61 ff.
This list is based on CrossRef data as of 7 september 2023. Please note that it may not be complete. Sources presented here have been supplied by the respective publishers.
Any errors therein should be reported to them.