Cover not available
Article published In:
FORUM
Vol. 22:1 (2024) ► pp.77–95

Exploring user perspectives

Student evaluation of the evolution in neural machine translation from English to Korean and its implications

 | Ewha Womans University
 | Dongguk University
Since the advent of Google NMT in 2016, human translators have been overwhelmed by the concern about being replaced by machine translation. Although professional translators argue that the machine translation output is not refined enough to surpass human translators, their claims are sometimes emotional and based on incorrect perceptions, without verification and substantiation from user evaluation and specific quality evaluation data. Therefore, this study examines the evolution of NMT output from English to Korean diachronically and provides specific user evaluation data that can verify and substantiate the claims. Despite the steady improvement in NMT performance observed in recent years, it has been recognized that there is still a significant gap that must be bridged for NMT to achieve parity with human translation. Nevertheless, as the collaboration between NMT and human translators is expected to increase, it is essential to reinforce relevant systems, such as technological and legal support systems.
Keywords: neural machine translation, human translation, translation evaluation, translation comparison, diachronic analysis
Article outline
  • 1.Introduction
  • 2.Fear spreading among human translators
  • 3.NMT user evaluation process
    • 3.1Overview of the evaluation
    • 3.2Evaluators
    • 3.3Procedures
    • 3.4Source texts for NMT
  • 4.Results of quantitative evaluation of NMT output
  • 5.Results of qualitative evaluation of NMT output
  • 6.Discussions and conclusion
  • Notes
  • References
Published online: 25 April 2024
DOI logo
https://doi.org/10.1075/forum.00036.shi
References (22)
References
Alonso, Elisa, and Lucas Nunes Vieira. 2017. “The Translator’s Amanuensis 2020.” JoSTrans: The Journal of Specialized Translation (28): 345–361.Google Scholar
Cadwell, Patrick, Sharon O’Brien, and Carlos S. C. Teixeira. 2018. “Resistance and Accommodation: Factors for the (Non-) Adoption of Machine Translation among Professional Translators.” Perspectives 26(3): 301–321. DOI logoGoogle Scholar
Calude, Andreea S. 2004. “Machine Translation of Various Text Genres.” The New Zealand Linguistic Society Journal (46): 67–94.Google Scholar
Google Translate. Accessed August 16, 2019. [URL]
. Accessed September 1, 2021. [URL]
Ha, Seonyoung. 2017. “‘Human vs AI’ Translation Battle...Human Beats the Machine. The Korea Daily, February 21. [URL]
Johnson. 2017. “Why Translators Have the Blues. A Profession under Pressure.” The Economist, May 24. [URL]
Kang, Insun. 2018. “In the Age of Artificial Intelligence and Robots...Translator/Casher will Disappear, Entertainer/Writer will Survive.” Maeil Business News Korea, April 2. [URL]
Marcus, G. 2018. The deepest problem with deep learning. Medium. Available at [URL]
Melby, Alan K. 2020. “Future of Machine Translation.” In The Routledge Handbook of Translation and Technology, edited by Minako O’Hagan. 419–436. London and New York: Routledge.Google Scholar
Moorkens, Joss. 2017. “Under Pressure: Translation in Times of Austerity.” Perspectives 25 (3): 464–477. DOI logoGoogle Scholar
O’Thomas, Mark. 2017. “Humanum ex Machina: Translation in the Post-global, Posthuman World.” Target 29(2): 284–300. DOI logoGoogle Scholar
Papago. Accessed August 16, 2019. [URL]
. Accessed September 1m 2021. [URL]
Park, Hyewon. 2022. “What is ‘This Job’ That is not Taken away by AI and Robots? Chosun Ilbo, April 15, [URL]
Pym, Anthony. 2020. “Translation, Risk Management, and Cognition.” In The Routledge Handbook of Translation and Cognition, edited by Arnt Lykke Jakobsen and Fabio Alves. 445–458. London and New York: Routledge. DOI logoGoogle Scholar
Seo, Bo-Hyun, and Soonyoung Kim. 2018. “An Analysis of Errors in Machine Translation.” Journal of Translation Studies 19(1): 99–116. DOI logoGoogle Scholar
Tonja, Atnafu Lambebo, Olga Kolesnikova, Alexander Gelbukh, and Grigori Sidorov. 2023. “Low-Resource Neural Machine Translation Improvement Using Source-Side Monolingual Data.” Applied Sciences 13(2): 1–20. DOI logoGoogle Scholar
Vieira, Lucas Nunes. 2020. “Automation Anxiety and Translators.” Translation Studies 13(1): 1–21. DOI logoGoogle Scholar
Vilar, David, Jia Xu, Luis Fernando D’Haro, and Hermann Ney. 2006. “Error analysis of Statistical Machine Translation Output.” The 5th International Conference on Language Resources and Evaluation, Genoa, ItalyGoogle Scholar
Zanzotto, Fabio Massimo. 2019. “Human-in-the-loop Artificial Intelligence.” Journal of Artificial Intelligence Research 641: 243–252. DOI logoGoogle Scholar
Zetzsche, Jost. 2020. “Freelance Translators’ Perspectives.” In The Routledge Handbook of Translation and Technology, edited by Minako O’Hagan. 166–182. London and New York: Routledge.Google Scholar