Translators’ perceptions of literary post-editing using statistical and neural machine translation
Joss Moorkens | Dublin City University
Antonio Toral | University of Groningen
Sheila Castilho | Dublin City University
Andy Way | Dublin City University
In the context of recent improvements in the quality of machine translation (MT) output and new use cases being found for that
output, this article reports on an experiment using statistical and neural MT systems to translate literature. Six professional
translators with experience of literary translation produced English-to-Catalan translations under three conditions: translation
from scratch, neural MT post-editing, and statistical MT post-editing. They provided feedback before and after the translation via
questionnaires and interviews. While all participants prefer to translate from scratch, mostly due to the freedom to be creative
without the constraints of segment-level segmentation, those with less experience find the MT suggestions useful.
Keywords: machine translation, literary machine translation, neural machine translation, human factors in machine translation
Article outline
- Introduction
- Related literature
- Perceptions of post-editing
- Neural MT
- MT for literature
- Methodology
- MT systems and training
- Test data: Warbreaker
- Participant profiles
- Experimental setup
- Results
- Measurements and perceptions of post-editing effort
- Temporal effort
- Technical effort
- Users’ perceptions of the task
- Creativity vs. productivity
- Segmentation and context
- Source vs. target (adequacy vs. fluency)
- SMT vs. NMT
- Perception vs. reality
- Alternative methods of machine assistance
- Measurements and perceptions of post-editing effort
- Conclusion
- Acknowledgements
- Notes
-
References
Published online: 28 November 2018
https://doi.org/10.1075/ts.18014.moo
https://doi.org/10.1075/ts.18014.moo
References
Aziz, Wilker, Sheila Castilho M. de Sousa, and Lucia Specia
2012 “PET: a tool for post-editing and assessing machine translation.” In The Eighth International Conference on Language Resources and Evaluation, LREC ‘12, Istanbul, Turkey. May 2012. http://www.lrec-conf.org/proceedings/lrec2012/pdf/985_Paper.pdf
Bahdanau, Dzmitry, Kyunghyun Cho, and Yoshua Bengio
2014 Neural Machine Translation by Jointly Learning to Align and Translate. Conference paper presented at the ICLR 2015. arXiv preprint, arXiv:1409.0473.
Bar-Hillel, Yehoshua
Bentivogli, Luisa, Arianna Bisazza, Mauro Cettolo, Marcello Federico
Besacier, Laurent
Bird, Steven
Bojar, Ondřej, Rajen Chatterjee, Christian Federmann, Yvette Graham, Barry Haddow, Matthias Huck, Antonio Jimeno Yepes, Philipp Koehn, Varvara Logacheva, Christof Monz, Matteo Negri, Aurélie Névéol, Mariana Neves, Martin Popel, Matt Post, Raphael Rubino, Carolina Scarton, Lucia Specia, Marco Turchi, Karin Verspoor, Marcos Zampieri
Bowker, Lynne
Cadwell, Patrick, Sharon O’Brien, and Carlos S. C. Teixeira
Cadwell, Patrick, Sheila Castilho, Sharon O’Brien, and Linda Mitchell
Carl, Michael, Silke Gutermuth, and Silvia Hansen-Schirra
Castilho, Sheila, and Sharon O’Brien
Castilho, Sheila, Joss Moorkens, Federico Gaspari, Iacer Calixto, John Tinsley, and Andy Way
Castilho, Sheila, Joss Moorkens, Federico Gaspari, Rico Sennrich, Vilelmini Sosoni, Panayota Georgakopoulou, Pintu Lohar, Andy Way, Antonio Valerio Miceli Barone, and Maria Gialama
2017 “A Comparative Quality Evaluation of PBSMT and NMT using Professional Translators.” Conference paper presented at the MT Summit 2017. Nagoya, Japan.
Catford, John C.
Church, Kenneth W., and Eduard H. Hovy
Daems, Joke, Orphée De Clercq, and Lieve Macken
De Almeida, Giselle, and Sharon O’Brien
Durrani, Nadir, Helmut Schmid, and Alexander Fraser
Forcada, Mikel L.
Gaspari, Federico, Antonio Toral, Sudip Kumar Naskar, Declan Groves, and Andy Way
Genzel, Dmitriy, Jakob Uszkoreit, and Franz Och
Green, Spence, Jeffrey Heer, and Christopher D. Manning
Greene, Erica, Tugba Bodrumlu, and Kevin Knight
Guerberof, Ana
Hassan, Hany, Anthony Aue, Chang Chen, Vishal Chowdhary, Jonathan Clark, Christian Federmann, Xuedong Huang, Marcin Junczys-Dowmunt, William Lewis, Mu Li, Shujie Liu, Tie-Yan Liu, Renqian Luo, Arul Menezes, Tao Qin, Frank Seide, Xu Tan, Fei Tian, Lijun Wu, Shuangzhi Wu, Yingce Xia, Dongdong Zhang, Zhirui Zhang, and Ming Zhou
Heyn, Matthias
Jones, Ruth, and Ann Irvine
2013 “The (Un)Faithful Machine Translator.” In Proceedings of the 7th Workshop on Language Technology for Cultural Heritage, Social Sciences, and Humanities, Sofia, Bulgaria, 96–101. Stroudsburg: Association for Computational Linguistics, http://www.aclweb.org/anthology/W13-2700
Kelly, Nataly
2014 “Why So Many Translators Hate Translation Technology.” Huffington Post. The Blog. https://www.huffingtonpost.com/nataly-kelly/why-so-many-translators-h_b_5506533.html
Klubička, Filip, Antonio Toral, and Víctor M. Sánchez-Cartagena
Koehn, P., and R. Knowles
2017 “Six Challenges for Neural Machine Translation.” In Proceedings of the First Workshop on Neural Machine Translation, Vancouver, BC, Canada, 28–39. http://www.aclweb.org/anthology/W17-3204. 
Koponen, Maarit
Krings, Hans P.
Lacruz, Isabel, and Gregory M. Shreve
LeBlanc, Matthieu
Lommel, Arle, and Donald A. DePalma
Martín, Juan Alberto Alonso, and Anna Civil Serra
Moorkens, Joss
Moorkens, Joss, and Sharon O’Brien
2015 “Post-Editing Evaluations: Trade-offs between Novice and Professional Participants.” In Proceedings of the 18th Annual Conference of the European Association for Machine Translation (EAMT 2015), edited by İIknur Durgar El-Kahlout, Mehmed Özkan, Felipe Sánchez-Martínez, Gema Ramírez-Sánchez, Fred Hollowood, and Andy Way, 75–81.
Moorkens, Joss, Sharon O’Brien, Igor A. L. Silva, Norma Fonseca, and Fabio Alves
Nitzke, Jean
PACTE group
Plitt, Mirko, and François Masselot
Pym, Anthony
Reiss, Katharina
Sennrich, Rico, Barry Haddow, and Alexandra Birch
Sennrich, Rico, Orhan Firat, Kyunghyun Cho, Alexandra Birch, Barry Haddow, Julian Hitschler, Marcin Junczys-Dowmunt, Samuel Läubli, Antonio Valerio Miceli Barone, Jozef Mokry, and Maria Nadejde
2017 “Nematus: A Toolkit for Neural Machine Translation.” In Proceedings of the Software Demonstrations from the 15th Conference of the European Chapter of the Association for Computational Linguistics, 65–68. http://aclweb.org/anthology/E17-3000. 
Snover, Matthew, Bonnie Dorr, Richard Schwartz, Linnea Micciulla, and John Makhoul
Specia, Lucia
Specia, Lucia, and Kashif Shah
Taivalkoski-Shilov, Kristiina
Teixeira, Carlos S. C.
Thouin, Benoît
Toral, Antonio, and Andy Way
Toral, Antonio, and Victor M. Sánchez-Cartagena
Toral, Antonio, Martijn Wieling, and Andy Way
Vasconcellos, Muriel
1985 “Machine Aids to Translation: A Holistic Scenario for Maximizing the Technology.” In Overcoming Language Barriers: The Human/Machine Relationship, Proceedings of the IV Annual Conference on Language and Communication held from December 13 to December 14, 1985 in New York, edited by Humphrey Tonkin, and Karen Johnson-Weiner, 27–34. New York: Center for Research and Documentation on World Problems.
Vaswani, Ashish, Yinggong Zhao, Victoria Fossum and David Chiang
Viera, Lucas Nunes
Wagner, Elizabeth
Way, Andy
Wu, Yonghui, Mike Schuster, Zhifeng Chen, Quoc V. Le, Mohammad Norouzi, Wolfgang Macherey, Maxim Krikun, Yuan Cao, Qin Gao, Klaus Macherey, Jeff Klingner, Apurva Shah, Melvin Johnson, Xiaobing Liu, Łukasz Kaiser, Stephan Gouws, Yoshikiyo Kato, Taku Kudo, Hideto Kazawa, Keith Stevens, George Kurian, Nishant Patil, Wei Wang, Cliff Young, Jason Smith, Jason Riesa, Alex Rudnick, Oriol Vinyals, Greg Corrado, Macduff Hughes, and Jeffrey Dean
2016 “Google’s Neural Machine Translation System: Bridging the Gap between Human and Machine Translation”. arXiv preprint 1609.08144, https://arxiv.org/abs/1609.08144
Cited by
Cited by 14 other publications
Abdel Latif, Muhammad M. M.
Chang-Soo Lee
Guerberof-Arenas, Ana & Antonio Toral
Jiménez-Crespo, Miguel A.
Kenny, Dorothy & Marion Winters
Lee, Changsoo
Nunes Vieira, Lucas & Elisa Alonso
O’Brien, Sharon & Maureen Ehrensberger-Dow
Ragni, Valentina & Lucas Nunes Vieira
Sakamoto, Akiko & Masaru Yamada
Slessor, Stephen
Tardel, Anke
Yang, Yanxia, Xiangling Wang & Qingqing Yuan
Zhao, Shengfang
This list is based on CrossRef data as of 11 november 2021. Please note that it may not be complete. Sources presented here have been supplied by the respective publishers. Any errors therein should be reported to them.