Cover not available
Article published In:
Developments in Cognitive Translation and Interpreting Studies
Edited by Kairong Xiao and Sandra L. Halverson
[Cognitive Linguistic Studies 8:2] 2021
► pp. 378–403
Dialogue interpreting, self-revision in translation and post-editing

Investigating post-editing effort

Does directionality play a role?

ORCID logo | Ionian University
ORCID logo | Ionian University
ORCID logo | Ionian University

The working environment of translators has changed significantly in recent decades, with post-editing (PE) emerging as a new trend in the human translation workflow, particularly following the advent of neural machine translation (NMT) and the improvement of the quality of the machine translation (MT) raw output especially at the level of fluency. In addition, the directionality axiom is increasingly being questioned with translators working from and into their first language both in the context of translation (Buchweitz and Alves 2006; Pavlović and Jensen 2009; Fonseca and Barbosa 2015; Hunziker Heeb 2015; Ferreira 2013, 2014; Ferreira et al. 2016; Feng 2017) and in the context of PE (Garcia 2011; Sánchez-Gijón and Torres-Hostench 2014; da Silva et al. 2017; Toledo Báez 2018). In this study we employ product- and process-oriented approaches to investigate directionality in PE in the English-Greek language pair. In particular, we compare the cognitive, temporal, and technical effort expended by translators for the full PE of NMT output in L1 (Greek) with the effort required for the full PE of NMT output in L2 (English), while we also analyze the quality of the final translation product. Our findings reveal that PE in L2, i.e., inverse PE, is less demanding than PE in L1, i.e., direct PE, in terms of the time and keystrokes required, and the cognitive load exerted on translators. Finally, our research shows that directionality does not imply differences in quality.

Keywords: directionality, cognitive effort, technical effort, temporal effort, translation quality
Article outline
  • 1.Introduction
  • 2.Methodology
    • 2.1Participants
    • 2.2STs and MT outputs
    • 2.3Process and product analysis
  • 3.Findings and discussion
    • 3.1Process analysis: Measuring PE effort
      • 3.1.1Temporal effort
      • 3.1.2Technical effort
      • 3.1.3Cognitive effort
    • 3.2Product analysis
  • 4.Conclusion
  • Acknowledgements
  • Notes
  • References
Published online: 22 November 2021
DOI logo
https://doi.org/10.1075/cogls.00083.sta
References
Akbari, A., & Segers, W.
(2017) Translation difficulty: How to measure and what to measure. Lebende Sprachen, 62(1), 3–29. DOI logoGoogle Scholar
Alves, F., Pagano, A., Neumann, S., Steiner, E., & Hansen-Schirra, S.
(2010) Translation units and grammatical shifts: Towards an integration of product- and process-based translation research. In G. M. Shreve & E. Angelone (Eds.), Translation and cognition (pp. 109–142). Amsterdam: John Benjamins. DOI logoGoogle Scholar
Beeby, L. A.
(2001) Direction of translation (directionality). In M. Baker (Ed.), Routledge encyclopedia of translation studies (pp. 63–67). London: Routledge.Google Scholar
Blum-Kulka, S.
(2000) Shifts of cohesion and coherence in translation. In L. Venuti (Ed.), The translation studies reader (pp. 298–312). London: Routledge.Google Scholar
Buchweitz, A., & Alves, F.
(2006) Cognitive adaptation in translation: an interface between language direction, time, and recursiveness in target text production. Letras de Hoje, 41(2), 241–272.Google Scholar
Campbell, S.
(1998) Translation into the second language. London: Longman.Google Scholar
Carl, M.
(2012) Translog–II: A program for recording user activity data for empirical reading and writing research. In Proceedings of the 8th international conference on language resources and evaluation (LREC’12) (pp. 4108–4112). Istanbul: European Language Resources Association (ELRA).Google Scholar
Carl, M., & Buch-Kromann, M.
(2010) Correlating translation product and translation process data of professional and student translators. In F. Yvon & V. Hansen (Eds.), Proceedings of the 14th annual conference of the European Association for Machine Translation (n.p.). Saint-Raphaël, France: European Association for Machine Translation.Google Scholar
Carl, M., Dragsted, B., Elming, J., Hardt, D., & Jakobsen, A. L.
(2011) The process of post-editing: a pilot study. Copenhagen Studies in Language, 411, 131–142.Google Scholar
Castilho, S., Moorkens, J., Gaspari, F., Calixto, I., Tinsley, J., & Way, A.
(2017a) Is neural machine translation the new state of the art? The Prague Bulletin of Mathematical Linguistics, 108(1), 109–120. DOI logoGoogle Scholar
Castilho, S., Moorkens, J., Gaspari, F., Sennrich, R., Sosoni, V., Georgakopoulou, Y., Lohar, P., Way, A., Miceli Barone, A., & Gialama, M.
(2017b) A Comparative quality evaluation of PBSMT and NMT using professional translators. In Proceedings of machine translation summit XVI (pp. 116–131). Nagoya, Japan.Google Scholar
Cohen, J.
(1960) A coefficient of agreement for nominal scales. Educational and Psychological Measurement, 20(1), 37–46. DOI logoGoogle Scholar
Conklin, K., Pellicer-Sánchez, A., & Carrol, G.
(2018) Eye-tracking: A guide for applied linguistics research. Cambridge: Cambridge University Press. DOI logoGoogle Scholar
Da Silva, I. A. L., Alves, F., Schmaltz, M., Pagano, A., Wong, D., Chao, L., Leal, A. L., Quaresma, P., Garcia, C., & Da Silva, G. E.
(2017) Translation, post-editing and directionality: A study of effort in the Chinese-Portuguese language pair. In A. L. Jakobsen & B. Mesa-Lao (Eds.), Translation in transition. Between cognition, computing and technology (pp. 108–134). Amsterdam: John Benjamins. DOI logoGoogle Scholar
Daems, J., Vandepitte, S., Hartsuiker, R. J., & Macken, L.
(2017) Identifying the machine translation error types with the greatest impact on post-editing effort. Frontiers in Psychology, 81, 1282. DOI logoGoogle Scholar
De Almeida, G.
(2013) Translating the post-editor: An investigation of post-editing changes and correlations with professional experience across two romance languages. (Unpublished doctoral dissertation, Dublin City University). Retrieved from [URL]
Densmer, L.
(2014, August 19). Light and full MT post-editing explained [RWS Moravia blog]. Retrieved from [URL]
Depraetere, I.
(2010) What counts as useful advice in a university post-editing training context? Report on a case study. In F. Yvon & V. Hansen (Eds.), Proceedings of the 14th annual conference of the European Association for Machine Translation (n.p.). Saint-Raphaël, France: European Association for Machine Translation.Google Scholar
Do Carmo, F.
(2019) Edit distances do not describe editing, but they can be useful for translation process research. In M. Carl & S. Hansen-Schirra (Eds.), Proceedings of the 2nd MEMENTO workshop on Modelling Parameters of Cognitive Effort in Translation Production (pp. 1–2). Dublin, Ireland: European Association for Machine Translation.Google Scholar
Dodds, J. M.
(1999) Friends, false friends and foes or back to basics in L1 to L2 translation. In G. Anderman & M. Rogers (Eds.), Word, text, translation: Liber Amicorum for Peter Newmark (pp. 56–65). Clevedon: Multilingual.Google Scholar
Doherty, S., & Kenny, D.
(2014) The design and evaluation of a statistical machine translation syllabus for translation students. The Interpreter and Translator Trainer, 8(2), 295–315. DOI logoGoogle Scholar
Doherty, S., Kenny, D., & Way, A.
(2012) Taking statistical machine translation to the student translator. In Proceedings of the tenth biennial conference of the Association for Machine Translation in the Americas (n.p.). San Diego, USA: Association for Machine Translation in the Americas.Google Scholar
Doherty, S., O’Brien, S., & Carl, M.
(2010) Eye tracking as an automatic MT evaluation technique. Machine Translation, 24(1), 1–13. DOI logoGoogle Scholar
Elming, J., Balling, L. W., & Carl, M.
(2014) Investigating user behaviour in post-editing and translation using the CASMACAT workbench. In S. O’Brien, L. Winther Balling, M. Carl, M. Simard & L. Specia (Eds.), Post-editing of machine translation (pp.147–169). Newcastle: Cambridge Scholars Publishing.Google Scholar
Feng, J.
(2017) Comparing cognitive load in L1 and L2 translation: Evidence from eye-tracking. Foreign Languages in China (《中国外语》), 781, 79–91.Google Scholar
Ferreira, A.
(2013) Direcionalidade em tradução: O papel da subcompetência bilíngue em tarefas de tradução L1 e L2. (Unpublished doctoral dissertation, Federal University of Minas Gerais). Retrieved from [URL]
Ferreira, A., & Schwieter, J. W.
(2017) Directionality in translation. In J. W. Schwieter & A. Ferreira (Eds.), The handbook of translation and cognition (pp. 90–105). Hoboken, NJ: Wiley-Blackwell. DOI logoGoogle Scholar
Ferreira, A., Schwieter, J. W., Gottardo, A., & Jones, J.
(2016) Cognitive effort in direct and inverse translation performance: insight from eye-tracking technology. Cadernos de Tradução, 36(3), 60–80. DOI logoGoogle Scholar
Firbas, J.
(1992) Functional sentence perspective in written and spoken communication (Studies in English Language). Cambridge: Cambridge University Press. DOI logoGoogle Scholar
Flanagan, M., & Christensen, T. P.
(2014) Testing post-editing guidelines: how translation trainees interpret them and how to tailor them for translator training purposes. The Interpreter and Translator Trainer, 8(2), 257–275. DOI logoGoogle Scholar
Fonseca, D. L., & Barbosa, N.
(2015) Directionality in translation: Investigating prototypical patterns in editing procedures. Translation & Interpreting, 7(1), 111–125.Google Scholar
Fridman, L., Langhans, P., Lee, J., & Reimer, B.
(2016) Driver gaze region estimation without use of eye movement. IEEE Intelligent Systems, 31(3), 49–56. DOI logoGoogle Scholar
Garcia, I.
(2011) Translating by post-editing: is it the way forward? Machine Translation, 25(3), 217–237. DOI logoGoogle Scholar
García-Yebra, V.
(1983) En torno a la traducción: Teoría, crítica, historia. Madrid: Gredos.Google Scholar
Gouadec, D.
(2007) Translator as a profession. Amsterdam: John Benjamins. DOI logoGoogle Scholar
Grosman, M., Kadrič, M., Kovačič, I., & Snell-Hornby, M.
(Eds.) (2000) Translation into non-mother tongues in professional practice and training. Tübingen: Stauffenburg.Google Scholar
Guerberof, A., & Moorkens, J.
(2019) Machine translation and post-editing training as part of a master’s programme. Jostrans: The Journal of Specialised Translation, 311, 217–238.Google Scholar
Halverson, S. L.
(2017) Multimethod approaches. In J. W. Schwieter & A. Ferreira (Eds.), The handbook of translation and cognition (pp. 195–212). Hoboken, NJ: Wiley-Blackwell. DOI logoGoogle Scholar
Hu, K., & Cadwell, P.
(2016) A comparative study of post-editing guidelines. Baltic Journal of Modern Computing, 4(2), 346–353.Google Scholar
Hunziker Heeb, A.
(2015) Does professional translation into L2 involve more effort than into L1? In Translation Process Research: Workshop 4 (n.p.). Las Palmas, Spain.Google Scholar
Hvelplund, K. T.
(2011) Allocation of cognitive resources in translation: An eye-tracking and key-logging study. (Doctoral dissertation, Copenhagen Business School). Frederiksberg: Samfundslitteratur.
Jia, Y., Carl, M., & Wang, X.
(2019) Post-editing neural machine translation versus phrase-based machine translation for English-Chinese. Machine Translation, 33(1–2), 9–29. DOI logoGoogle Scholar
Kenny, D., & Doherty, S.
(2014) Statistical machine translation in the translation curriculum: Overcoming obstacles and empowering translators. The Interpreter and Translator Trainer, 8(2), 276–294. DOI logoGoogle Scholar
Koponen, M.
(2012) Comparing human perceptions of post-editing effort with post-editing operations. In Proceedings of the seventh workshop on statistical machine translation (pp. 181–190). Montréal, Canada: Association for Computational Linguistics.Google Scholar
(2015) How to teach machine translation post-editing? Experiences from a post-editing course. In S. O’Brien & M. Simard (Eds.), Proceedings of the 4th workshop on post-editing technology and practice (WPTP4) (pp. 2–15). Miami, FA: Association for Machine Translation in the Americas.Google Scholar
Koponen, M., & Salmi, L.
(2017) Post-editing quality: Analysing the correctness and necessity of post-editor corrections. Linguistica Antverpiensia, New Series: Themes in Translation Studies, 161, 137–148.Google Scholar
Koponen, M., Salmi, L., & Nikulin, M.
(2019) A product and process analysis of post-editor corrections on neural, statistical and rule-based machine translation output. Machine Translation, 331, 61–90. DOI logoGoogle Scholar
Krings, Η.
(2001) Repairing texts: Empirical investigations of machine translation post-editing processes. Kent, OH: Kent State University Press.Google Scholar
Lacruz, I., & Shreve, G. M.
(2014) Pauses and cognitive effort in post-editing. In S. O’Brien, L. Winther Balling, M. Carl, M. Simard, & L. Specia (Eds.), Post-editing of machine translation (pp. 246–272). Newcastle: Cambridge Scholars Publishing.Google Scholar
Ladmiral, J. R.
(1979) Théorèmes pour la Traduction. Paris: Didier.Google Scholar
Lavie, A., & Agarwal, A.
(2007) METEOR: An Automatic Metric for MT Evaluation with High Levels of Correlation with Human Judgments. In Proceedings of the Second Workshop on Statistical Machine Translation. Association for Computational Linguistics (pp. 228–231). Prague, Czech Republic: Association for Computational Linguistics. DOI logoGoogle Scholar
Massardo, I., van der Meer, J., O’Brien, S., Hollowood, F., Aranberri, N., & Drescher, K.
(2016) MT post-editing guidelines. The Netherlands: TAUS Signature Editions.Google Scholar
Mesa-Lao, B.
(2012) The next generation translator’s workbench: post-editing in CASMACAT v. 1.0. In Proceedings of the 34th Translating and the Computer Conference (n.p.). London, UK: ASLIBGoogle Scholar
(2014) Gaze behaviour on source texts: An exploratory study comparing translation and post-editing. In S. O’Brien, L. Winther Balling, M. Carl, M. Simard, & L. Specia (Eds.), Post-editing of machine translation (pp. 219–245). Newcastle: Cambridge Scholars Publishing.Google Scholar
Monti, J., Seretan, V., Corpas-Pastor, G., & Mitkov, R.
(2018) Multiword units in machine translation and translation technology. In R. Mitkov, J. Monti, G. C. Pastor & V. Seretan (Eds.), Multiword units in machine translation and translation technology (pp. 1–40). Amsterdam: John Benjamins. DOI logoGoogle Scholar
Moorkens, J., & O’Brien, S.
(2015) Post-editing evaluations: trade-offs between novice and professional participants. In Proceedings of the 18th Annual Conference of the European Association for Machine Translation (pp. 75–81). Antalya, Turkey: European Association for Machine Translation.Google Scholar
Moorkens, J., O’Brien, S., da Silva, I. A. L., Fonseca, N., & Alves, F.
(2015) Correlations of perceived post-editing effort with measurements of actual effort. Machine Translation, 291, 267–284. DOI logoGoogle Scholar
Mráček, D.
(2018) Inverse translation: the more challenging direction. Linguistica Pragensia, 28(2), 202–221.Google Scholar
Newmark, P.
(1981) Approaches to translation. Oxford: Pergamon.Google Scholar
(1988) A Textbook of translation. London: Prentice Hall.Google Scholar
Nitzke, J.
(2019) Problem solving activities in post-editing and translation from scratch: A multi-method study. Berlin: Language Science Press.Google Scholar
O’Brien, S.
(2002) Teaching post-editing: A proposal for course content. In Proceedings of the 6th EAMT Workshop: Teaching machine translation (pp. 99–106). Manchester, UK: European Association for Machine Translation.Google Scholar
(2009) Eye tracking in translation process research: methodological challenges and solutions. Methodology, technology and innovation in translation process research, 381, 251–266.Google Scholar
(2010) Introduction to post-editing: Who, what, how and where to next. In The Ninth Conference of the Association for Machine Translation in the Americas. Denver, Colorado: Association for Machine Translation in the Americas.Google Scholar
O’Brien, S., & Simard, M.
(2014) Introduction to special issue on post-editing. Machine Translation, 28(3), 159–164. DOI logoGoogle Scholar
Papineni, K., Roukos, S., Ward, T., & Zhu, W. J.
(2002) BLEU: a method for automatic evaluation of machine translation. In Proceedings of the 40th annual meeting on association for computational linguistics (311–318). Philadelphia, Pennsylvania, USA: Association for Computational Linguistics.Google Scholar
Pavlović, N.
(2007) Directionality in Collaborative Translation Processes. A Study of Novice Translators. (Unpublished doctoral dissertation, Universitat Rovira i Virgili). Retrieved from [URL]
Pavlović, N., & Jensen, K. T. H.
(2009) Eye tracking translation directionality. In A. Pym & A. Perekrestenko (Eds.), Translation research projects 21 (Vol. 21, pp. 93–109). Tarragona: Intercultural Studies Group.Google Scholar
Pokorn, N.
(2005) Challenging the traditional axioms: Translation into a non-mother tongue. Amsterdam: John Benjamins. DOI logoGoogle Scholar
Popovic, M., Arčan, M., & Lommel, A.
(2016) Potential and limits of using post-edits as reference translations for mt evaluation. Baltic Journal of Modern Computing 2 (Vol. 4, No. 2, pp. 218–229). Riga, Latvia.Google Scholar
Raptis, S., & Giagkou, M.
(2016) From capturing to generating human behavior: closing the interaction loop at the hubic lab. In Proceedings of the 20th panhellenic conference on informatics (PCI) (pp. 1–6). Partas, Greece: ACM Digital Library, International Conference Proceedings Series. DOI logoGoogle Scholar
Sánchez-Gijón, P., & Torres-Hostench, O.
(2014) MT Post-editing into the mother tongue or into a foreign language? Spanish-to-English MT translation output post-edited by translation trainees. In S. O’Brien, M. Simard & L. Specia (Eds.), Proceedings of the Third Workshop on Post-Editing Technology and Practice (WPTP-3)-AMTA Workshop (pp. 15–19). Vancouver, Canada: Association for Machine Translation in the Americas.Google Scholar
Schmitt, P. A.
(1998) Marktsituation der Übersetzer. In M. Snell-Hornby, H. G. Hönig, P. Kussmaul & P. A. Schmitt (Eds.), Handbuch translation (pp. 5–13). Tübingen: Stauffenburg.Google Scholar
Sharmin, S., Spakov, O., Räihä, K. J., & Jakobsen, A. L.
(2008) Where on the screen do translation students look while translating, and for how long? Copenhagen studies in language, 361, 31–51.Google Scholar
Shreve, G., & Angelone, Ε.
(Εds.) (2010) Translation and cognition. Amsterdam: John Benjamins. DOI logoGoogle Scholar
Snover, M., Dorr, B., Schwartz, R., Micciulla, L., & Makhoul, J.
(2006) A study of translation edit rate with targeted human annotation. In Proceedings of the 7th Conference of the Association for Machine Translation in the Americas (Vol. 200. No. 6, pp. 223–231). Cambridge, Massachusetts, USA: The Association for Machine Translation in the Americas.Google Scholar
Talaván, N., & Rodríguez-Arancón, P.
(2014) The use of reverse subtitling as an online collaborative language learning tool. The Interpreter and Translator Trainer, 8(1), 84–101. DOI logoGoogle Scholar
Toledo Báez, M. C.
(2018) Machine translation and post-editing: Impact of training and directionality on quality and productivity. Revista Tradumàtica. Technologies de la Traducció, 161, 24–34. DOI logoGoogle Scholar
Vidal, E.
(1997) Finite-state speech-to-speech translation. In 1997 IEEE International Conference on Acoustics, Speech, and Signal Processing (Vol. 11, pp. 111–114). Munich, Germany: IEEE Computer Society. DOI logoGoogle Scholar
Vieira, L. N., Alonso, E., & Bywood, L.
(2019) Introduction: Post-editing in practice – Process, product and networks. Jostrans: The Journal of Specialised Translation, 311, 2–13.Google Scholar
Wang, B.
(2011) Translation practices and the issue of directionality in China. Meta, 56(4), 896–914. DOI logoGoogle Scholar
Whyatt, B.
(2019) In search of directionality effects in the translation process and in the end product. Translation, Cognition and Behaviour, 2(1), 79–100. DOI logoGoogle Scholar
Xiao, K.
(2013) Cognitive approaches to translation process: Current trends, challenges and future development. In R. A. Awang, A. A. G. Ghani & L. A. Ayob (Eds.), Translator and Interpreter Education and Training: Innovation, Assessment and Recognition (pp.432–448). Kuala Lumpur: Malaysian Translators Association.Google Scholar