Cover not available
Article published In:
FORUM
Vol. 21:1 (2023) ► pp.139–162

Cognitive effort in human translation and machine translation post-editing processes

A holistic and phased view

| Faculty of Languages and Linguistics, Universiti Malaya
ORCID logo | Faculty of Languages and Linguistics, Universiti Malaya
Due to the inaccuracies of Machine Translation (MT), Post-Editing (PE) is inevitable. This poses questions over whether human effort to polish an MT is worthwhile or whether it would be more efficient to translate manually. However, to date, fewer attempts have been made to compare the cognitive effort in the PE process and the sub-phases (orientation, drafting, and revision) of PE with that of Human Translation (HT). To fill this gap, the current study aims to investigate and compare cognitive effort in HT and PE processes in translation from Chinese to English. Data were collected via eye-tracking and keyboard-logging approaches from 25 participants recruited to fulfil three HT and three PE tasks respectively. The comparison of cognitive effort was made from the processes of HT and PE, and their different sub-phases. The study reveals a significant difference in cognitive effort, orientation duration, and drafting duration between HT and PE.
Keywords: machine translation, post-editing, cognitive effort, translation process
Article outline
  • 1.Introduction
  • 2.Related research
    • 2.1Cognitive effort in the human translation process
    • 2.2PE effort
    • 2.3Pause
  • 3.Method
    • 3.1Research design
    • 3.2Participants
    • 3.3Materials
    • 3.4Data quality
    • 3.5Data analysis
  • 4.Findings
    • 4.1Cognitive effort in HT and PE: Whole process
      • 4.1.1Total task time
      • 4.1.2Total fixation duration
      • 4.1.3Fixation count
      • 4.1.4Pauses
      • 4.1.5Keyboard activities
    • 4.2Cognitive effort in HT and PE: Phased view
      • 4.2.1Orientation
      • 4.2.2Drafting
      • 4.2.3Revision
  • 5.Discussion and conclusion
  • Acknowledgements
  • Notes
  • References
Published online: 6 July 2023
DOI logo
https://doi.org/10.1075/forum.22009.wan
References (57)
References
Alves, Fabio, Arlene Koglin, Bartolomé Mesa-Lao, Mercedes García Martínez, Norma B. de Lima Fonseca, Arthur de Melo Sá, José Luiz Gonçalves, Karina Sarto Szpak, Kyoko Sekino, and Marceli Aquino. 2016. “Analysing the Impact of Interactive Machine Translation on Post-Editing Effort.” In New Directions in Empirical Translation Process Research, edited by Michael Carl, Srinivas Bangalore, and Moritz Schaeffer, 1st ed., 77–94. Cham: Springer. DOI logoGoogle Scholar
Bradley, James V. 1958. “Complete Counterbalancing of Immediate Sequential Effects in a Latin Square Design.” Journal of the American Statistical Association 53 (282): 525–28. DOI logoGoogle Scholar
Carl, Michael. 2012. “The CRITT TPR-DB 1.0: A Database for Empirical Human Translation Process Research.” In Workshop on Post-Editing Technology and Practice, 9–18. San Diego, California, USA: Association for Machine Translation in the Americas. [URL]
Carl, Michael, Barbara Dragsted, Jakob Elming, Daniel Hardt, and Arnt Lykke Jakobsen. 2011. “The Process of Post-Editing: A Pilot Study.” Copenhagen Studies in Language 411: 131–42.Google Scholar
Carl, Michael, Barbara Dragsted, and Arnt Lykke Jakobsen. 2011. “On the Systematicity of Human Translation Processes.” In Tralogy 2011. Translation Careers and Technologies: Convergence Points for the Future. Paris, France. [URL]
Carl, Michael, and Martin Kay. 2011. “Gazing and Typing Activities during Translation: A Comparative Study of Translation Units of Professional and Student Translators.” Meta 56 (4): 952–75. DOI logoGoogle Scholar
Conklin, Kathy, Ana Pellicer-Sánchez, and Gareth Carrol. 2018. Eye-Tracking: A Guide for Applied Linguistics Research. Cambridge, UK: Cambridge University Press. DOI logoGoogle Scholar
Cumbreno, Cristina, and Nora Aranberri. 2019. “Comparison of Temporal, Technical and Cognitive Dimension Measurements for Post-Editing Effort.” In Second MEMENTO Workshop on Modelling Parameters of Cognitive Effort in Translation Production, 5–6. Dublin, Ireland.Google Scholar
Daems, Joke, Sonia Vandepitte, Robert J. Hartsuiker, and Lieve Macken. 2017. “Translation Methods and Experience: A Comparative Analysis of Human Translation and Post-Editing with Students and Professional Translators.” Meta 62 (2): 245–70. DOI logoGoogle Scholar
Dragsted, Barbara. 2010. “Coordination of Reading and Writing Processes in Translation.” In Translation and Cognition, edited by Gregory M. Shreve and Erik Angelone, 41–61. Philadelphia, USA: John Benjamins Publishing. DOI logoGoogle Scholar
Dragsted, Barbara, and Michael Carl. 2013. “Towards a Classification of Translation Styles Based on Eye-Tracking and Keylogging Data.” Journal of Writing Research 5 (1): 133–58. DOI logoGoogle Scholar
Durban, Chris. 2011. “Translation – Getting It Right. A Guide to Buying Translation.” American Translators Association. Accessed April 22, 2022. [URL]
Ferreira, Aline, John Wayne Schwieter, Alexandra Gottardo, and Jefferey Jones. 2016. “Cognitive Effort in Direct and Inverse Translation Performance: Insight from Eye-Tracking Technology.” Cadernos de Tradução 36 (3): 60–80. DOI logoGoogle Scholar
Green, Spence, Jeffrey Heer, and Christopher D. Manning. 2013. “The Efficacy of Human Post-Editing for Language Translation.” In SIGCHI Conference on Human Factors in Computing Systems, 439–48. Paris, France. DOI logoGoogle Scholar
Guerberof-Arenas, Ana. 2012. Productivity and Quality in the Post-Editing of Outputs from Translation Memories and Machine Translation. PhD diss. Universitat Rovira I Virgili.
. 2014. “The Role of Professional Experience in Post-Editing from a Quality and Productivity Perspective.” In Post-Editing of Machine Translation: Processes and Applications, edited by Laura Winther Balling, Michael Carl, Sharon O’Brien, Michel Simard, and Specia Lucia, 51–77. Newcastle-upon-Tyne, UK: Cambridge Scholars Publishing.Google Scholar
Hvelplund, Kristian Tangsgaard. 2011. Allocation of Cognitive Resources in Translation. An Eye-Tracking and Key-Logging Study.Google Scholar
. 2014. “Eye Tracking and the Translation Process: Reflections on the Analysis and Interpretation of Eye-Tracking Data.” MonTI Special Issue – Minding Translation, 201–23. DOI logoGoogle Scholar
Jakobsen, Arnt Lykke. 2002. “Translation Drafting by Professional Translators and by Translation Students.” Copenhagen Studies in Language 271: 191–204.Google Scholar
Jia, Yanfang, Michael Carl, and Xiangling Wang. 2019. “Post-Editing Neural Machine Translation versus Phrase-Based Machine Translation for English–Chinese.” Machine Translation 33 (1): 9–29. DOI logoGoogle Scholar
Just, M., and P. Carpenter. 1976. “Eye Fixations and Cognitive Processes.” Cognitive Psychology, 8 (4): 441–80. DOI logoGoogle Scholar
Just, Marcel Adam, and Patricia A. Carpenter. 1980. “A Theory of Reading: From Eye Fixations to Comprehension.” Psychological Review 871: 329–54. DOI logoGoogle Scholar
Koglin, Arlene. 2015. “An Empirical Investigation of Cognitive Effort Required to Post-Edit Machine Translated Metaphors Compared to the Translation of Metaphors.” Translation & Interpreting 7 (1): 126–41. DOI logoGoogle Scholar
Koglin, Arlene, and Rossana Cunha. 2019. “Investigating the Post-Editing Effort Associated with Machine-Translated Metaphors: A Process-Driven Analysis.” Journal of Specialised Translation 311: 38–59.Google Scholar
Koponen, Maarit. 2012. “Comparing Human Perceptions of Post-Editing Effort with Post-Editing Operations.” In Seventh Workshop on Statistical Machine Translation, 181–90. Montreal, Canada: Association for Computational Linguistics. [URL]
. 2016. “Is Machine Translation Post-Editing Worth the Effort? A Survey of Research into Post-Editing and Effort.” Journal of Specialised Translation 251: 131–48.Google Scholar
Koponen, Maarit, Wilker Aziz, Luciana Ramos, and Lucia Specia. 2012. “Post-editing time as a measure of cognitive effort.“ In AMTA 2012 Workshop on Post-editing Technology and Practice (WPTP). San Diego, California: Association for Machine Translation in the Americas. [URL]
Krings, Hans P. 2001. Repairing Texts: Empirical Investigations of Machine Translation Post-Editing Processes. Kent, Ohio, USA: Kent State University Press.Google Scholar
Lacruz, Isabel, Michael Carl, Masaru Yamada, and Akiko Aizawa. 2016. “Pause Metrics and Machine Translation Utility.” In The 22nd Annual Meeting of the Association for Natural Language Processing, NLP 2016, 1213–16. Sendai, Japan: The Association for Natural Language Processing.Google Scholar
Lacruz, Isabel, Michael Denkowski, and Alon Lavie. 2014. “Cognitive Demand and Cognitive Effort in Post-Editing.” In Proceedings of the 11th Conference of the Association for Machine Translation in the Americas, 73–84. Vancouver, Canada: Association for Machine Translation in the Americas. [URL]
Lacruz, Isabel, and Gregory M. Shreve. 2014. “Pauses and Cognitive Effort in Post-Editing.” In Post-Editing of Machine Translation: Processes and Applications, edited by Laura Winther Balling, Michael Carl, and Sharon O’Brien, 246–73. Newcastle upon Tyne, UK: Cambridge Scholars Publishing.Google Scholar
Lacruz, Isabel, Gregory M. Shreve, and Erik Angelone. 2012. “Average Pause Ratio as an Indicator of Cognitive Effort in Post-Editing: A Case Study.” In Workshop on Post-Editing Technology and Practice. San Diego, California, USA: Association for Machine Translation in the Americas. [URL]
Läubli, Samuel, Chantal Amrhein, Patrick Düggelin, Beatriz Gonzalez, Alena Zwahlen, and Martin Volk. 2019. “Post-Editing Productivity with Neural Machine Translation: An Empirical Assessment of Speed and Quality in the Banking and Finance Domain.” ArXiv Preprint. arxiv.org/abs/1906.01685.Google Scholar
Martín, Ricardo Muñoz. 2010. “On Paradigms and Cognitive Translatology.” In Translation and Cognition, edited by Gregory M. Shreve and Erik Angelone, 169–87. Philadelphia, Pa, USA: John Benjamins Publishing. DOI logoGoogle Scholar
Mesa-Lao, Bartolomé. 2014. “Gaze Behaviour on Source Texts: An Exploratory Study Comparing Translation and Post-Editing.” In Post-Editing of Machine Translation: Processes and Applications, 219–45. Cambridge, UK: Cambridge Scholars Publishing.Google Scholar
Nitzke, Jean, and Katharina Oster. 2016. “Comparing Translation and Post-Editing: An Annotation Schema for Activity Units.” In New Directions in Empirical Translation Process Research, edited by Michael Carl, Srinivas Bangalore, and Moritz Schaeffer, 293–308. DOI logoGoogle Scholar
O’Brien, Sharon. 2006. “Pauses as Indicators of Cognitive Effort in Post-Editing Machine Translation Output.” Across Languages and Cultures 7 (1): 1–21. DOI logoGoogle Scholar
. 2007. “An Empirical Investigation of Temporal and Technical Post-Editing Effort.” Translation and Interpreting Studies 2 (1): 83–136. DOI logoGoogle Scholar
Pellicer-Sánchez, Ana. 2016. “Incidental L2 Vocabulary Acquisition from and While Reading: An Eye-Tracking Study.” Studies in Second Language Acquisition 381: 97–130. DOI logoGoogle Scholar
Plitt, Mirko, and François Masselot. 2010. “A Productivity Test of Statistical Machine Translation Post-Editing in a Typical Localisation Context.” The Prague Bulletin of Mathematical Linguistics 931: 7–16. DOI logoGoogle Scholar
R Core Team. 2021. “R: A Language and Environment for Statistical Computing.” Vienna, Austria: R Foundation for Statistical Computing. [URL]
Richardson, John T. E. 2018. “The Use of Latin-Square Designs in Educational and Psychological Research.” Educational Research Review 241: 84–97. DOI logoGoogle Scholar
Sanchez-Torron, Marina, and Philipp Koehn. 2016. “Machine Translation Quality and Post-Editor Productivity.” In AMTA 2016 – The Twelfth Conference of the Association for Machine Translation in the Americas, edited by Spence Green and Lane Schwartz, 11:16–26. Austin, Texas, USA: Association for Machine Translation in the Americas. [URL]
Schaeffer, Moritz, Michael Carl, Isabel Lacruz, and Akiko Aizawa. 2016. “Measuring Cognitive Translation Effort with Activity Units.” Baltic Journal of Modern Computing 4 (2): 331–45.Google Scholar
Screen, Benjamin. 2017. “Productivity and Quality When Editing Machine Translation and Translation Memory Outputs: An Empirical Analysis of English to Welsh Translation.” Studia Celtica Posnaniensia 2 (1): 119–42. DOI logoGoogle Scholar
Sjørup, Annette Camilla. 2013. Cognitive Effort in Metaphor Translation: An Eye-Tracking and Key-Logging Study. PhD diss. Copenhagen Business School.
Sousa, Sheila C. M. de, Wilker Aziz, and Lucia Specia. 2011. “Assessing the Post-Editing Effort for Automatic and Semi-Automatic Translations of DVD Subtitles.” In Recent Advances in Natural Language Processing, edited by Ruslan Mitkov and Galia Angelova, 97–103. Hissar, Bulgaria.Google Scholar
Stasimioti, Maria, and Vilelmini Sosoni. 2020. “Translation vs Post-Editing of NMT Output: Measuring Effort in the English-Greek Language Pair.” In 14th Conference of the Association for Machine Translation in the Americas,1st Workshop on Post-Editing in Modern-Day Translation, 109–24.Google Scholar
. 2021. “Investigating Post-Editing: A Mixed-Methods Study with Experienced and Novice Translators in the English-Greek Language Pair.” In Translation, Interpreting, Cognition: The Way out of the Box, edited by Tra&Co Group, 79–104. Berlin, Germany: Language Science Press. DOI logoGoogle Scholar
Sun, Sanjun, and Gregory M. Shreve. 2014. “Measuring Translation Difficulty.” Target 26 (1): 98–127. DOI logoGoogle Scholar
Sung, Yao-Ting, Tao-Hsing Chang, Wei-Chun Lin, Kuan-Sheng Hsieh, and Kuo-En Chang. 2016. “CRIE: An Automated Analyzer for Chinese Texts.” Behavior Research Methods 48 (4): 1238–51. DOI logoGoogle Scholar
Vanroy, Bram, Orphée De Clercq, and Lieve Macken. 2019. “Correlating Process and Product Data to Get an Insight into Translation Difficulty.” Perspectives: Studies in Translation Theory and Practice 27 (6): 924–41. DOI logoGoogle Scholar
Vieira, Lucas Nunes. 2014. “Indices of Cognitive Effort in Machine Translation Post-Editing.” Machine Translation 281: 187–216. DOI logoGoogle Scholar
Whyatt, Bogusława. 2019. “In Search of Directionality Effects in the Translation Process and in the End Product.” Translation, Cognition & Behavior 2 (1): 79–100. DOI logoGoogle Scholar
Whyatt, Bogusława, Katarzyna Stachowiak, and Marta Kajzer-Wietrzny. 2016. “Similar and Different: Cognitive Rhythm and Effort in Translation and Paraphrasing.” Poznan Studies in Contemporary Linguistics 52 (2): 175–208. DOI logoGoogle Scholar
Whyatt, Bogusława, Olga Witczak, and Ewa Tomczak. 2021. “Information Behaviour in Bidirectional Translators: Focus on Online Resources.” Interpreter and Translator Trainer 15 (2): 154–71. DOI logoGoogle Scholar
Zhechev, Ventsislav. 2012. “Machine Translation Infrastructure and Post-Editing Performance at Autodesk.” In Workshop on Post-Editing Technology and Practice. San Diego, California: Association for Machine Translation in the Americas. [URL]
Cited by (1)

Cited by one other publication

Shi, Lei
2023. Algorithmic Translation Correction Mechanisms: An End-to-end Algorithmic Implementation of English-Chinese Machine Translation. ICST Transactions on Scalable Information Systems DOI logo

This list is based on CrossRef data as of 4 july 2024. Please note that it may not be complete. Sources presented here have been supplied by the respective publishers. Any errors therein should be reported to them.