Article published in:
Translation Spaces
Vol. 7:2 (2018) ► pp. 240–262

Translators’ perceptions of literary post-editing using statistical and neural machine translation

| Dublin City University
| University of Groningen
| Dublin City University
| Dublin City University
In the context of recent improvements in the quality of machine translation (MT) output and new use cases being found for that output, this article reports on an experiment using statistical and neural MT systems to translate literature. Six professional translators with experience of literary translation produced English-to-Catalan translations under three conditions: translation from scratch, neural MT post-editing, and statistical MT post-editing. They provided feedback before and after the translation via questionnaires and interviews. While all participants prefer to translate from scratch, mostly due to the freedom to be creative without the constraints of segment-level segmentation, those with less experience find the MT suggestions useful.
Keywords: machine translation, literary machine translation, neural machine translation, human factors in machine translation
Article outline
  • Introduction
  • Related literature
    • Perceptions of post-editing
    • Neural MT
    • MT for literature
  • Methodology
    • MT systems and training
    • Test data: Warbreaker
    • Participant profiles
    • Experimental setup
  • Results
    • Measurements and perceptions of post-editing effort
      • Temporal effort
      • Technical effort
    • Users’ perceptions of the task
      • Creativity vs. productivity
      • Segmentation and context
      • Source vs. target (adequacy vs. fluency)
      • SMT vs. NMT
      • Perception vs. reality
      • Alternative methods of machine assistance
  • Conclusion
  • Acknowledgements
  • Notes
  • References
Published online: 28 November 2018
https://doi.org/10.1075/ts.18014.moo
References
Aziz, Wilker, Sheila Castilho M. de Sousa, and Lucia Specia
2012 “PET: a tool for post-editing and assessing machine translation.” In The Eighth International Conference on Language Resources and Evaluation, LREC ‘12, Istanbul, Turkey. May 2012. http://​www​.lrec​-conf​.org​/proceedings​/lrec2012​/pdf​/985​_Paper​.pdf
Bahdanau, Dzmitry, Kyunghyun Cho, and Yoshua Bengio
2014 Neural Machine Translation by Jointly Learning to Align and Translate. Conference paper presented at the ICLR 2015. arXiv preprint, arXiv:1409.0473.
Bar-Hillel, Yehoshua
1960 “The Present Status of Automatic Translation of Languages.” Advances in Computers 11: 91–163. CrossrefGoogle Scholar
Bentivogli, Luisa, Arianna Bisazza, Mauro Cettolo, Marcello Federico
2017 “Neural versus Phrase-Based MT Quality: An In-Depth Analysis on English-German and English-French”. Computer Speech & Language 491: 52–70. CrossrefGoogle Scholar
Besacier, Laurent
2014 “Traduction automatisée d’une œuvre littéraire: une étude pilote.” In Traitement Automatique du Langage Naturel (TALN). Marseille, France.Google Scholar
Bird, Steven
2006 “NLTK: The Natural Language Toolkit.” In Proceedings of the COLING/ACL on Interactive Presentation Sessions, 69–72. Sydney, Australia. CrossrefGoogle Scholar
Bojar, Ondřej, Rajen Chatterjee, Christian Federmann, Yvette Graham, Barry Haddow, Matthias Huck, Antonio Jimeno Yepes, Philipp Koehn, Varvara Logacheva, Christof Monz, Matteo Negri, Aurélie Névéol, Mariana Neves, Martin Popel, Matt Post, Raphael Rubino, Carolina Scarton, Lucia Specia, Marco Turchi, Karin Verspoor, Marcos Zampieri
2016 “Findings of the 2016 Conference on Machine Translation (WMT16).” In Proceedings of the First Conference on Machine Translation 21: 131–198. Berlin, Germany: Association for Computational Linguistics.Google Scholar
Bowker, Lynne
2007 “Translation Memory and ‘Text’.” In Lexicography, Terminology, and Translation. Text-Based Studies in Honour of Ingrid Meyer, edited by Lynne Bowker, 175–187. Ottawa: University of Ottawa Press.Google Scholar
Cadwell, Patrick, Sharon O’Brien, and Carlos S. C. Teixeira
2017 “Resistance and Accommodation: Factors for the (Non-) Adoption of Machine Translation among Professional Translators.” Perspectives: Studies in Translation Theory and Practice 26 (3): 301–321. CrossrefGoogle Scholar
Cadwell, Patrick, Sheila Castilho, Sharon O’Brien, and Linda Mitchell
2016 “Human Factors in Machine Translation and Post-Editing Among Institutional Translators.” Translation Spaces 5 (2): 222–243. CrossrefGoogle Scholar
Carl, Michael, Silke Gutermuth, and Silvia Hansen-Schirra
2015 “Post-Editing Machine Translation: A Usability Test for Professional Translation Settings.” In Psycholinguistic and Cognitive Inquiries into Translation and Interpreting, edited by Aline Ferreira and John W. Schwieter, 145–174. Amsterdam: John Benjamins. CrossrefGoogle Scholar
Castilho, Sheila, and Sharon O’Brien
2016 “Content Profiling and Translation Scenarios.” The Journal of Internationalization and Localization 3(1): 18–37. CrossrefGoogle Scholar
Castilho, Sheila, Joss Moorkens, Federico Gaspari, Iacer Calixto, John Tinsley, and Andy Way
2017 “Is Neural Machine Translation the New State of the Art?” The Prague Bulletin of Mathematical Linguistics 1081: 109–120. CrossrefGoogle Scholar
Castilho, Sheila, Joss Moorkens, Federico Gaspari, Rico Sennrich, Vilelmini Sosoni, Panayota Georgakopoulou, Pintu Lohar, Andy Way, Antonio Valerio Miceli Barone, and Maria Gialama
2017 “A Comparative Quality Evaluation of PBSMT and NMT using Professional Translators.” Conference paper presented at the MT Summit 2017. Nagoya, Japan.
Catford, John C.
1965 A Linguistic Theory of Translation: An Essay in Applied Linguistics. London: Oxford University Press.Google Scholar
Church, Kenneth W., and Eduard H. Hovy
1993 “Good Applications for Crummy Machine Translation.” Machine Translation 8 (4): 239–258. CrossrefGoogle Scholar
Daems, Joke, Orphée De Clercq, and Lieve Macken
2017 “Translationese and Post-Editese: How Comparable is Comparable Quality?” Linguistica Antverpiensia, New Series: Themes in Translation Studies 161: 89–103.Google Scholar
De Almeida, Giselle, and Sharon O’Brien
2010 “Analysing Post-Editing Performance: Correlations with Years of Translation Experience.” In Proceedings of the 14th Annual Conference of the European Association for Machine Translation held in St. Raphaël, France.Google Scholar
Durrani, Nadir, Helmut Schmid, and Alexander Fraser
2011 “A Joint Sequence Translation Model with Integrated Reordering.” In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, from June 19 to 24, in Portland, Oregon, 1045–1054.Google Scholar
Forcada, Mikel L.
2017 “Making Sense of Neural Machine Translation.” Translation Spaces 6 (2): 291–309. CrossrefGoogle Scholar
Gaspari, Federico, Antonio Toral, Sudip Kumar Naskar, Declan Groves, and Andy Way
2014 “Perception vs Reality: Measuring Machine Translation Post-Editing Productivity.” In Proceedings of AMTA 2014 Workshop on Post-editing Technology and Practice, Vancouver, 60–72.Google Scholar
Genzel, Dmitriy, Jakob Uszkoreit, and Franz Och
2010 “ ‘Poetic’ Statistical Machine Translation: Rhyme and Meter.” In the EMNLP ’10 Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing, Cambridge, Massachusetts, 158–166. Stroudsburg: Association for Computational Linguistics.Google Scholar
Green, Spence, Jeffrey Heer, and Christopher D. Manning
2013 “The Efficacy of Human Post-Editing for Language Translation.” In the CHI ’13 Proceedings of the SIGCHI Conference Factors in Computing Systems. New York: ACM Press. CrossrefGoogle Scholar
Greene, Erica, Tugba Bodrumlu, and Kevin Knight
2010 “Automatic Analysis of Rhythmic Poetry with Applications to Generation and Translation.” In the EMNLP ’10 Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing, Cambridge, Massachusetts, 524–533. Stroudsburg: Association for Computational Linguistics.Google Scholar
Guerberof, Ana
2012 “Productivity and Quality in the Post-Editing of Outputs from Translation Memories and Machine Translation.” PhD Dissertation. Universitat Rovira i Virgili.Google Scholar
Hassan, Hany, Anthony Aue, Chang Chen, Vishal Chowdhary, Jonathan Clark, Christian Federmann, Xuedong Huang, Marcin Junczys-Dowmunt, William Lewis, Mu Li, Shujie Liu, Tie-Yan Liu, Renqian Luo, Arul Menezes, Tao Qin, Frank Seide, Xu Tan, Fei Tian, Lijun Wu, Shuangzhi Wu, Yingce Xia, Dongdong Zhang, Zhirui Zhang, and Ming Zhou
2018 “Achieving Human Parity on Automatic Chinese to English News Translation.” Redmond: Microsoft AI & Research. arXiv:1803.05567.Google Scholar
Heyn, Matthias
1998 “Translation Memories: Insights and Prospects.” In Unity in Diversity? Current Trends in Translation Studies, edited by Lynne Bowker, Michael Cronin, Dorothy Kenny, and Jennifer Pearson, 123–36. Manchester: St. Jerome Publishing.Google Scholar
Jones, Ruth, and Ann Irvine
2013 “The (Un)Faithful Machine Translator.” In Proceedings of the 7th Workshop on Language Technology for Cultural Heritage, Social Sciences, and Humanities, Sofia, Bulgaria, 96–101. Stroudsburg: Association for Computational Linguistics, http://​www​.aclweb​.org​/anthology​/W13​-2700
Kelly, Nataly
2014 “Why So Many Translators Hate Translation Technology.” Huffington Post. The Blog. https://​www​.huffingtonpost​.com​/nataly​-kelly​/why​-so​-many​-translators​-h​_b​_5506533​.html
Klubička, Filip, Antonio Toral, and Víctor M. Sánchez-Cartagena
2017 “Fine-Grained Human Evaluation of Neural Versus Phrase-Based Machine Translation.” The Prague Bulletin of Mathematical Linguistics 1081: 121–132. CrossrefGoogle Scholar
Koehn, P., and R. Knowles
2017 “Six Challenges for Neural Machine Translation.” In Proceedings of the First Workshop on Neural Machine Translation, Vancouver, BC, Canada, 28–39. http://​www​.aclweb​.org​/anthology​/W17​-3204. Crossref
Koponen, Maarit
2016 “Is Post-Editing Worth the Effort? A Survey of Research into Post-Editing and Effort.” JosTrans: Journal of Specialised Translation 251: 131–148.Google Scholar
Krings, Hans P.
2001 Repairing Texts: Empirical Investigations of Machine Translation Post-Editing Processes. Ohio: Kent State University Press.Google Scholar
Lacruz, Isabel, and Gregory M. Shreve
2014 “Pauses and Cognitive Effort in Post-Editing.” In Post-Editing of Machine Translation: Processes and Applications, edited by Sharon O’Brien, Laura Winther Balling, Michael Carl, Michel Simard, and Lucia Specia, 287–314. Newcastle-Upon-Tyne: Cambridge Scholars.Google Scholar
LeBlanc, Matthieu
2013 “Translators on Translation Memory (TM). Results of an Ethnographic Study in Three Translation Services and Agencies.” The International Journal for Translation and Interpreting Research 5 (2): 1–13. CrossrefGoogle Scholar
Lommel, Arle, and Donald A. DePalma
2016 “Europe’s Leading Role in Machine Translation: How Europe Is Driving the Shift to MT.” Technical Report. Boston: Common Sense Advisory.Google Scholar
Martín, Juan Alberto Alonso, and Anna Civil Serra
2014 “Integration of a Machine Translation System into the Editorial Process Flow of a Daily Newspaper.” Procesamiento del Lenguaje Natural Revista 531: 193–196.Google Scholar
Moorkens, Joss
2017 “Under Pressure: Translation in Times of Austerity.” Perspectives 25 (3): 464–477. CrossrefGoogle Scholar
Moorkens, Joss, and Sharon O’Brien
2015 “Post-Editing Evaluations: Trade-offs between Novice and Professional Participants.” In Proceedings of the 18th Annual Conference of the European Association for Machine Translation (EAMT 2015), edited by İIknur Durgar El-Kahlout, Mehmed Özkan, Felipe Sánchez-Martínez, Gema Ramírez-Sánchez, Fred Hollowood, and Andy Way, 75–81.Google Scholar
2017 “Assessing User Interface Needs of Post-Editors of Machine Translation.” In Human Issues in Translation Technology: The IATIS Yearbook, edited by Dorothy Kenny, 109–130, Oxford, United Kingdom: Routledge.Google Scholar
Moorkens, Joss, Sharon O’Brien, Igor A. L. Silva, Norma Fonseca, and Fabio Alves
2015 “Correlations of perceived post-editing effort with measurements of actual effort.” Machine Translation 29 (3–4): 267–284. CrossrefGoogle Scholar
Nida, Eugene
1964 Towards a Science of Translating. Leiden: Brill.Google Scholar
Nitzke, Jean
2016 “Monolingual Post-Editing: An Exploratory Study on Research Behaviour and Target Text Quality.” In Eye-tracking and Applied Linguistics, edited by Silvia Hansen-Schirra, and Sambor Grucza, 83–109. Berlin: Language Science Press. CrossrefGoogle Scholar
PACTE group
2005 “Investigating Translation Competence: Conceptual and Methodological Issues.” Meta 50 (2): 609–619. CrossrefGoogle Scholar
Plitt, Mirko, and François Masselot
2010 “A Productivity Test of Statistical Machine Translation Post-Editing in a Typical Localisation Context.” The Prague Bulletin of Mathematical Linguistics 931: 7–16. CrossrefGoogle Scholar
Pym, Anthony
2008 “Professional Corpora: Teaching Strategies for Work with Online Documentation, Translation Memories and Content Management.” Chinese Translator’s Journal 29 (2): 41–45.Google Scholar
Reiss, Katharina
1981 “Type, Kind and Individuality of Text: Decision Making in Translation.” Poetics Today 2 (4): 121–131. CrossrefGoogle Scholar
Sennrich, Rico, Barry Haddow, and Alexandra Birch
2016a “Improving Neural Machine Translation Models with Monolingual Data.” In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics from August 7 to August 12, 2016, 86–96. Berlin, Germany.Google Scholar
2016b “Neural Machine Translation of Rare Words with Subword Units.” In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics from August 7 to August 12, 2016, 1715–1725. Berlin, Germany.Google Scholar
2016c “Controlling Politeness in Neural Machine Translation via Side Constraints.” In Proceedings of NAACL-HLT 2016, 351–40.Google Scholar
Sennrich, Rico, Orhan Firat, Kyunghyun Cho, Alexandra Birch, Barry Haddow, Julian Hitschler, Marcin Junczys-Dowmunt, Samuel Läubli, Antonio Valerio Miceli Barone, Jozef Mokry, and Maria Nadejde
2017 “Nematus: A Toolkit for Neural Machine Translation.” In Proceedings of the Software Demonstrations from the 15th Conference of the European Chapter of the Association for Computational Linguistics, 65–68. http://​aclweb​.org​/anthology​/E17​-3000. Crossref
Somers, Harold
2001 Computers and Translation: A Translator’s Guide. Amsterdam: John Benjamins. CrossrefGoogle Scholar
Snover, Matthew, Bonnie Dorr, Richard Schwartz, Linnea Micciulla, and John Makhoul
2006 “A Study of Translation Edit Rate with Targeted Human Annotation.” Proceedings of Association for Machine Translation in the Americas.Google Scholar
Specia, Lucia
2011 “Exploiting Objective Annotations for Measuring Translation Post-Editing Effort.” In Proceedings of the 15th Conference of the European Association for Machine Translation, 73–80. Leuven, Belgium.Google Scholar
Specia, Lucia, and Kashif Shah
2018 “Machine Translation Quality Estimation: Applications and Future Perspectives.” In Translation Quality Assessment: From Principles to Practice, edited by Joss Moorkens, Sheila Castilho, Federico Gaspari, and Stephen Doherty, 201–236. Heidelberg: Springer International Publishing. CrossrefGoogle Scholar
Taivalkoski-Shilov, Kristiina
2018 “Ethical Issues Regarding Machine(-assisted) Translation of Literary Texts.” Perspectives: Studies in Translation Theory and Practice (online first). Special Issue: Voice, Translation, and Ethics, ed. by Cecilia Alvstad, Annjo K. Greenall, Hanne Jansen, and Kristiina Taivalkoski-Shilov. CrossrefGoogle Scholar
Teixeira, Carlos S. C.
2014 “Perceived vs. Measured Performance in the Post-Editing of Suggestions from Machine Translation and Translation Memories.” In Proceedings of the Third Workshop on Post-Editing Technology and Practice (WPTP-3), edited by Sharon O’Brien, Michel Simard, and Lucia Specia, 45–59.Google Scholar
Thouin, Benoît
1982 “The METEO System.” In Practical Experience of Machine Translation: Proceedings of Translating and the Computer 1981, edited by Veronica Lawson, 39–44, Amsterdam: North-Holland Publishing.Google Scholar
Toral, Antonio, and Andy Way
2015 “Translating Literary Text between Related Languages using SMT.” In Proceedings of NAACL-HLT Fourth Workshop on Computational Linguistics for Literature, 123–132. Denver, Colorado. CrossrefGoogle Scholar
2018 “What level of quality can Neural Machine Translation attain on literary text?” In Translation Quality Assessment: From Principles to Practice, edited by Joss Moorkens, Sheila Castilho, Federico Gaspari, and Stephen Doherty, 263–287. Heidelberg: Springer International Publishing. CrossrefGoogle Scholar
Toral, Antonio, and Victor M. Sánchez-Cartagena
2017 “A Multifaceted Evaluation of Neural versus Phrase-Based Machine Translation for 9 Language Directions.” In Conference of the European Chapter of the Association for Computational Linguistics, EACL 2017. Valencia, Spain. CrossrefGoogle Scholar
Toral, Antonio, Martijn Wieling, and Andy Way
2018 “Post-editing Effort of a Novel with Statistical and Neural Machine Translation.” Frontiers in Digital Humanities 51:9. CrossrefGoogle Scholar
Vasconcellos, Muriel
1985 “Machine Aids to Translation: A Holistic Scenario for Maximizing the Technology.” In Overcoming Language Barriers: The Human/Machine Relationship, Proceedings of the IV Annual Conference on Language and Communication held from December 13 to December 14, 1985 in New York, edited by Humphrey Tonkin, and Karen Johnson-Weiner, 27–34. New York: Center for Research and Documentation on World Problems.Google Scholar
Vaswani, Ashish, Yinggong Zhao, Victoria Fossum and David Chiang
2013 “Decoding with Large-Scale Neural Language Models Improves Translation.” In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, Seattle, Washington, 1387–1392. Stroudsburg: Association for Computational Linguistics.Google Scholar
Viera, Lucas Nunes
2014 “Indices of cognitive effort in machine translation post-editing.” Machine Translation 28 (3–4):187–216. CrossrefGoogle Scholar
Wagner, Elizabeth
1985 “Post-Editing Systran-A Challenge for Commission Translators.” Terminologie et Traduction 31: 1–7.Google Scholar
Way, Andy
2013 “Traditional and Emerging Use-Cases for Machine Translation.” In Proceedings of Translating and the Computer 351. London, United Kingdom.Google Scholar
2018a “Quality Expectations of Machine Translation.” In Translation Quality Assessment: From Principles to Practice, edited by Joss Moorkens, Sheila Castilho, Federico Gaspari, and Stephen Doherty, 159–178. Heidelberg: Springer International Publishing. CrossrefGoogle Scholar
2018b “Machine Translation: Where We Are at Today.” In The Bloomsbury Companion to Language Industry Studies, edited by Erik Angelone, Gary Massey, and Maureen Ehrensberger-Dow. London: Bloomsbury.Google Scholar
Wu, Yonghui, Mike Schuster, Zhifeng Chen, Quoc V. Le, Mohammad Norouzi, Wolfgang Macherey, Maxim Krikun, Yuan Cao, Qin Gao, Klaus Macherey, Jeff Klingner, Apurva Shah, Melvin Johnson, Xiaobing Liu, Łukasz Kaiser, Stephan Gouws, Yoshikiyo Kato, Taku Kudo, Hideto Kazawa, Keith Stevens, George Kurian, Nishant Patil, Wei Wang, Cliff Young, Jason Smith, Jason Riesa, Alex Rudnick, Oriol Vinyals, Greg Corrado, Macduff Hughes, and Jeffrey Dean
2016 “Google’s Neural Machine Translation System: Bridging the Gap between Human and Machine Translation”. arXiv preprint 1609.08144, https://​arxiv​.org​/abs​/1609​.08144
Cited by

Cited by 14 other publications

Abdel Latif, Muhammad M. M.
2020.  In Translator and Interpreter Education Research [New Frontiers in Translation Studies, ], ► pp. 125 ff. Crossref logo
Chang-Soo Lee
2019. 문학번역에서의 기계번역과 인간번역 문체에 대한 전산문체학적 비교 연구. The Journal of Translation Studies 20:2 ► pp. 111 ff. Crossref logo
Guerberof-Arenas, Ana & Antonio Toral
2020. The impact of post-editing and machine translation on creativity and reading experience. Translation Spaces 9:2 ► pp. 255 ff. Crossref logo
Jiménez-Crespo, Miguel A.
2020. The “technological turn” in translation studies. Translation Spaces 9:2 ► pp. 314 ff. Crossref logo
Kenny, Dorothy & Marion Winters
2020. Machine translation, ethics and the literary translator’s voice. Translation Spaces 9:1 ► pp. 123 ff. Crossref logo
Lee, Changsoo
2021. How do machine translators measure up to human literary translators in stylometric tests?. Digital Scholarship in the Humanities Crossref logo
Nunes Vieira, Lucas & Elisa Alonso
2020. Translating perceptions and managing expectations: an analysis of management and production perspectives on machine translation. Perspectives 28:2 ► pp. 163 ff. Crossref logo
O’Brien, Sharon & Maureen Ehrensberger-Dow
2020. MT Literacy—A cognitive view. Translation, Cognition & Behavior 3:2 ► pp. 145 ff. Crossref logo
Ragni, Valentina & Lucas Nunes Vieira
2021. What has changed with neural machine translation? A critical review of human factors. Perspectives ► pp. 1 ff. Crossref logo
Sakamoto, Akiko & Masaru Yamada
2020. Social groups in machine translationpost-editing. Translation Spaces 9:1 ► pp. 78 ff. Crossref logo
Slessor, Stephen
2020. Tenacious technophobes or nascent technophiles? A survey of the technological practices and needs of literary translators. Perspectives 28:2 ► pp. 238 ff. Crossref logo
Tardel, Anke
2021.  In Explorations in Empirical Translation Process Research [Machine Translation: Technologies and Applications, 3], ► pp. 81 ff. Crossref logo
Yang, Yanxia, Xiangling Wang & Qingqing Yuan
2021. Measuring the usability of machine translation in the classroom context. Translation and Interpreting Studies 16:1 ► pp. 101 ff. Crossref logo
Zhao, Shengfang
2021.  In Diverse Voices in Chinese Translation and Interpreting [New Frontiers in Translation Studies, ], ► pp. 399 ff. Crossref logo

This list is based on CrossRef data as of 11 november 2021. Please note that it may not be complete. Sources presented here have been supplied by the respective publishers. Any errors therein should be reported to them.