A competence matrix for machine translation-oriented data literacy teaching

Ralph Krüger and Janiça Hackenbuchner
TH Köln – University of Applied Sciences | Ghent University
Abstract

This article presents a matrix of competence descriptors aimed at machine translation-oriented data literacy teaching. This competence matrix constitutes the didactics-facing side of the DataLitMT project, which develops learning resources for teaching relevant components of data literacy in their translation-specific form of professional machine translation (MT) literacy to BA and MA students in translation and specialised communication. After highlighting the increasing relevance of both professional MT literacy and data literacy in the context of Translation Studies and professional translation, the article presents and discusses a professional MT literacy framework and an MT-specific data literacy framework, which serve to structure the two frames of reference relevant to this article. Then, the article provides a detailed discussion of the competence matrix developed based on the two frameworks sketched previously. This discussion is intended to show how the individual dimensions and sub-dimensions of data literacy were linked to relevant (sub-)dimensions of professional MT literacy and translated into corresponding competence descriptors. To conclude, the article presents an example of a learning resource for MT-oriented data literacy teaching developed based on the descriptors of the competence matrix.

Keywords:
Publication history

Date received: 6 August 2022

Date accepted: 17 March 2024

Published online: 9 April 2024

Table of contents

Since its introduction in production systems such as Google Translate in 2016, neural machine translation (NMT) has quickly become the state-of-the art machine translation (MT) paradigm, surpassing its predecessor phrase-based statistical MT in terms of overall translation quality (e.g., Toral and Sánchez-Cartagena 2017; Bentivogli et al. 2018). Within the NMT paradigm, a shift from recurrent to transformer neural networks (Vaswani et al. 2017) led to further qualitative improvements, prompting some MT developers to voice claims concerning human parity (Hassan et al. 2018) or even superhuman performance (Popel et al. 2020) of their NMT systems for particular language combinations and domains. Although these strong claims regarding NMT performance have been refuted by various authors (e.g., Läubli et al. 2020), the high base quality achievable by NMT systems has led this translation technology to become a dominant force both in the professional translation industry and in Translation Studies in recent years. NMT is impacting, for example, the cognitive processes of translators working in NMT-assisted translation production networks (e.g., Carl and Schaeffer 2017, 98ff.), the didactics of translation (technology) teaching (e.g., Ginovart Cid and Colominas Ventura 2021), business processes in the translation industry (e.g., TAUS 2018) and corresponding job profiles (e.g., European Union Institutions 2019, 8–9), as well as overall societal perceptions of the translation profession (e.g., Moorkens 2022, 129).

Full-text access is restricted to subscribers. Log in to obtain additional credentials. For subscription information see Subscription & Price. Direct PDF access to this article can be purchased through our e-platform.

References

Bentivogli, Luisa, Arianna Bisazza, Mauro Cettolo, and Marcello Federico
2018 “Neural versus Phrase-Based MT Quality: An in-Depth Analysis on English–German and English–French.” Computer Speech & Language 49: 52–70. DOI logoGoogle Scholar
Bloom, Benjamin S.
1956 “Taxonomy of Educational Objectives: The Classification of Educational Goals; Handbook I: Cognitive Domain.” In Taxonomy of Educational Objectives: The Classification of Educational Goals; Handbook I: Cognitive Domain, edited by Benjamin S. Bloom, Max D. Engelhart, Edward J. Furst, Walker H. Hill, and David R. Krathwohl, 1–61. New York: Longman.Google Scholar
Bowker, Lynne, and Jairo Buitrago Ciro
2019 Machine Translation and Global Research: Towards Improved Machine Translation Literacy in the Scholarly Community. Bingley: Emerald Publishing.Google Scholar
Bowker, Lynne
2021 “Machine Translation Use Outside the Language Industries: A Comparison of Five Delivery Formats for Machine Translation Literacy Instruction.” In TRanslation and Interpreting Technology ONline. Proceedings of the Conference, edited by Ruslan Mitkov, Vilhelmini Sosoni, Julie Christine Giguère, Elena Murgolo, and Elizabeth Deysel, 25–36. Shoumen: Incoma Ltd. http://​triton​-conference​.org​/proceedings/ DOI logo
Buysschaert, Joost, María Fernández-Parra, Koen Kerremans, Maarit Koponen, and Gys-Walt van Egdom
2018 “Embracing Digital Disruption in Translator Training: Technology Immersion in Simulated Translation Bureaus.” Revista Tradumàtica 16: 125–133.Google Scholar
Carl, Michael, and Barbara Dragsted
2017 “Inside the Monitor Model: Processes of Default and Challenged Translation Production.” In Crossroads between Contrastive Linguistics, Translation Studies and Machine Translation: TC3-II, edited by Oliver Czulo and Silvia Hansen-Schirra, 5–30. Berlin: Language Science Press. https://​langsci​-press​.org​/catalog​/view​/102​/191​/861​-3
Carl, Michael, and Moritz Schaeffer
2017 “Sketch of a Noisy Channel Model for the Translation Process.” In Empirical Modelling of Translation and Interpreting, edited by Silvia Hansen-Schirra, Oliver Czulo, and Sascha Hofmann, 71–116. Berlin: Language Science Press. https://​langsci​-press​.org​/catalog​/book​/132
Christensen, Tina Paulsen, Marian Flanagan, and Anne Schjoldager
2017 “Mapping Translation Technology Research in Translation Studies: An Introduction to the Thematic Section.” Hermes – Journal of Language and Communication in Business 56: 7–20. DOI logoGoogle Scholar
Council of Europe
2020 Common European Framework of Reference for Languages: Learning, Teaching, Assessment – Companion Volume. Strasbourg: Council of Europe Publishing.Google Scholar
DataLitMT
2023 “DataLitMT – Teaching Data Literacy in the Context of Machine Translation Literacy.” Project website. https://​itmk​.github​.io​/The​-DataLitMT​-Project/
Do Carmo, Félix, and Joss Moorkens
2021 “Differentiating Editing, Post-Editing and Revision.” In Translation, Revision and Post-Editing: Industry Practices and Cognitive Processes, edited by Maarit Koponen, Brian Mossop, Isabelle Robert, and Giovanna Scocchera, 35–49. Abingdon: Routledge.Google Scholar
Dorst, Aletta G., Susana Valdez, and Heather Bouman
2022 “Machine Translation in the Multilingual Classroom: How, When and Why Do Humanities Students at a Dutch University Use Machine Translation?” Translation and Translanguaging in Multilingual Contexts 8 (1): 49–66. DOI logoGoogle Scholar
ELIS Research
2022 European Language Industry Survey 2022. https://​elis​-survey​.org
EMT
2022 “European Master’s in Translation Competence Framework 2022.” Website of the DG Translation of the European Commission. https://​commission​.europa​.eu​/system​/files​/2022​-11​/emt​_competence​_fwk​_2022​_en​.pdf
European Commission
n. d. “Description of the Eight EQF Levels.” https://​europa​.eu​/europass​/en​/description​-eight​-eqf​-levels
Ginovart Cid, Clara, and Carme Colominas Ventura
2021 “The MT Post-Editing Skill Set: Course Descriptions and Educators’ Thoughts.” In Translation Revision and Post-Editing: Industry Practices and Cognitive Processes, edited by Maarit Koponen, Brian Mossop, Isabelle S. Robert, and Giovanna Scocchera, 226–246. Abingdon: Routledge.Google Scholar
Göpferich, Susanne
2008 Translationsprozessforschung: Stand – Methoden – Perspektiven [Translation process research: State of the art – methods – perspectives]. Tübingen: Narr.Google Scholar
Hassan, Hany, Anthony Aue, Chang Chen, Vishal Chowdhary, Jonathan Clark, Christian Federmann, Xuedong Huang, Marcin Junczys-Dowmunt, William Lewis, Mu Li, Shujie Liu, Tie-Yan Liu, Renqian Luo, Arul Menezes, Tao Qin, Frank Seide, Xu Tan, Fei Tian, Lijun Wu, Shuangzhi Wu, Yingce Xia, Dongdong Zhang, Zhirui Zhang, and Ming Zhou
2018 “Achieving Human Parity on Automatic Chinese to English News Translation.” arXiv. https://​arxiv​.org​/abs​/1803​.05567
Kenny, Dorothy, and Stephen Doherty
2014 “Statistical Machine Translation in the Translation Curriculum: Overcoming Obstacles and Empowering Translators.” The Interpreter and Translator Trainer 8 (2): 276–294. DOI logoGoogle Scholar
Kenny, Dorothy
2019 “Machine Translation.” In The Routledge Handbook of Translation and Philosophy, edited by Piers Rawling and Philip Wilson, 428–445. Abingdon: Routledge.Google Scholar
2020 “Technology and Translator Training.” In The Routledge Handbook of Translation and Technology, edited by Minako O’Hagan, 498–515. Abingdon: Routledge.Google Scholar
2022a “Introduction.” In Kenny (2022b, v–viii).Google Scholar
ed. 2022b Machine Translation for Everyone. Empowering Users in the Age of Artificial Intelligence. Berlin: Language Science Press.Google Scholar
Kiraly, Don
2013 “Towards a View of Translator Competence as an Emergent Phenomenon: Thinking Outside the Box(es) in Translator Education.” In New Prospects and Perspectives for Educating Language Mediators, edited by Don Kiraly, Silvia Hansen-Schirra, and Karin Maksymski, 197–224. Tübingen: Narr.Google Scholar
Koehn, Philipp
2020 Neural Machine Translation. Cambridge: Cambridge University Press. DOI logoGoogle Scholar
Krüger, Ralph
2022a “Integrating Professional Machine Translation Literacy and Data Literacy.” Lebende Sprachen 67 (2): 247–282. DOI logoGoogle Scholar
2022b “Using Jupyter Notebooks as Didactic Instruments in Translation Technology Teaching.” The Interpreter and Translator Trainer 16 (4): 503–523. DOI logoGoogle Scholar
Läubli, Samuel, Sheila Castilho, Graham Neubig, Rico Sennrich, Qinlan Shen, and Antonio Toral
2020 “A Set of Recommendations for Assessing Human-Machine Parity in Language Translation.” Journal of Artificial Intelligence Research 67: 653–672. DOI logoGoogle Scholar
Long, Duri, and Brian Magerko
2020 “What is AI Literacy? Competencies and Design Considerations.” In CHI ’20: Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems, edited by Regina Bernhaupt, Florian Mueller, David Verweij, and Josh Andres, 1–16. New York: Association for Computing Machinery. DOI logoGoogle Scholar
Loock, Rudy, and Sophie Léchauguette
2022 “Machine Translation Literacy and Undergraduate Students in Applied Languages: Report on an Exploratory Study.” Revista Tradumàtica 19: 204–225.Google Scholar
Misra, Archita
2021 “Advancing Data Literacy in the Post-Pandemic World: A Primer to Catalyse Policy Dialogue and Action.” Draft discussion paper for the 2021 PARIS21 Annual Meetings Data as a Public Good: Building Resilience for a Post-Pandemic World . PARIS21. https://​paris21​.org​/sites​/default​/files​/inline​-files​/DataLiteracy​_Primer​_0​.pdf
Moorkens, Joss, and Dave Lewis
2019 “Research Questions and a Proposal for the Future Governance of Translation Data.” Journal of Specialised Translation 32: 2–25.Google Scholar
Moorkens, Joss, and Marta Rocchi
2021 “Ethics in the Translation Industry.” In The Routledge Handbook of Translation and Ethics, edited by Kaisa Koskinen and Nike K. Pokorn, 320–337. Abingdon: Routledge.Google Scholar
Moorkens, Joss
2018 “What to Expect from Neural Machine Translation: A Practical In-Class Translation Evaluation Exercise.” The Interpreter and Translator Trainer 12 (4): 375–387. DOI logoGoogle Scholar
2022 “Ethics and Machine Translation.” In Kenny (2022b, 121–140).Google Scholar
Nitzke, Jean, Silvia Hansen-Schirra, and Carmen Canfora
2019 “Risk Management and Post-Editing Competence.” Journal of Specialised Translation 31: 239–259.Google Scholar
O’Brien, Sharon, Maureen Ehrensberger-Dow, Marcel Hasler, and Megan Conolly
2017 “Irritating CAT Tool Features that Matter to Translators.” Hermes – Journal of Language and Communication in Business 56: 145–162. DOI logoGoogle Scholar
Olohan, Maeve
2017 “Technology, Translation and Society.” Target 29 (2): 264–283. DOI logoGoogle Scholar
PACTE Group (Amparo Hurtado Albir, Anabel Galán-Mañas, Anna Kuznik, Christian Olalla-Soler, Patricia Rodríguez-Inés, and Lupe Romero
) 2018 “Competence Levels in Translation: Working Towards a European Framework.” The Interpreter and Translator Trainer 12 (2): 111–131. DOI logoGoogle Scholar
Popel, Martin, Marketa Tomkova, Jakub Tomek, Łukasz Kaiser, Jakob Uszkoreit, Ondřej Bojar, and Zdeněk Žabokrtský
2020 “Transforming Machine Translation: A Deep Learning System Reaches News Translation Quality Comparable to Human Professionals.” Nature Communications 11: 1–15. DOI logoGoogle Scholar
Ramírez-Sánchez, Gema
2022 “Custom Machine Translation.” In Kenny (2022b, 165–186).Google Scholar
Ridsdale, Chantel, James Rothwell, Mike Smit, Hossam Ali-Hassan, Michael Bliemel, Dean Irvine, Daniel Kelley, Stan Matwin, and Brad Wuetherick
2015 Strategies and Best Practices for Data Literacy Education: Knowledge Synthesis Report. Dalhousie University. http://​hdl​.handle​.net​/10222​/64578
Rupcic, Kerstin
2021 Einsatzpotenziale maschineller Übersetzung in der juristischen Fachübersetzung [Potential uses of machine translation in legal translation]. Berlin: Frank & Timme.Google Scholar
Schüller, Katharina
2020 Working Paper No. 53 – Future Skills: A Framework for Data Literacy. Essen: Hochschulforum Digitalisierung. https://​hochschulforumdigitalisierung​.de​/de​/future​-skills​-framework​-data​-literacy
Stanovsky, Gabriel, Noah A. Smith, and Luke Zettlemoyer
2019 “Evaluating Gender Bias in Machine Translation.” In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, edited by Anna Korhonen, David Traum, and Lluís Màrquez, 1679–1684. Florence: Association for Computational Linguistics. https://​aclanthology​.org​/P19​-1164/ DOI logo
TAUS
2018 “Translators in the Algorithmic Age: A Briefing Based on the Takeaways from TAUS Industry Leaders Forum ’18.” TAUS. https://​info​.taus​.net​/translators​-in​-the​-algorithmic​-age​-no​-time​-to​-read
Toral, Antonio, and Víctor M. Sánchez-Cartagena
2017 “A Multifaceted Evaluation of Neural versus Phrase-Based Machine Translation for 9 Language Directions.” In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 1, Long Papers, edited by Mirella Lapata, Phil Blunsom, and Alexander Koller, 1063–1073. Valencia: Association for Computational Linguistics. https://​aclanthology​.org​/E17​-1100/ DOI logo
Toral, Antonio
2019 “Post-Editese: An Exacerbated Translationese.” In Proceedings of Machine Translation Summit XVII: Research Track, edited by Mikel Forcada, Andy Way, Barry Haddow, and Rico Sennrich, 273–281. Dublin: European Association for Machine Translation. https://​aclanthology​.org​/W19​-6627/
Vanmassenhove, Eva, Dimitar Shterionov, and Matthew Gwilliam
2021 “Machine Translationese: Effects of Algorithmic Bias on Linguistic Complexity in Machine Translation.” In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, edited by Paola Merlo, Jörg Tiedemann, and Reut Tsarfaty, 2203–2213. Online: Association for Computational Linguistics. DOI logoGoogle Scholar
Vanmassenhove, Eva, Dimitar Shterionov, and Andy Way
2019 “Lost in Translation: Loss and Decay of Linguistic Richness in Machine Translation.” In Proceedings of Machine Translation Summit XVII: Research Track, edited by Mikel L. Forcada, Andy Way, Barry Haddow, and Rico Sennrich, 222–232. Dublin: European Association for Machine Translation.Google Scholar
Vaswani, Ashish, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Łukasz Kaiser, and Illia Polosukhin
2017 “Attention Is All You Need.” In Advances in Neural Information Processing Systems 30 (NIPS 2017), edited by Isabelle Guyon, Ulrike von Luxburg, Samy Bengio, Hanna M. Wallach, Rob Fergus, S. V. N. Vishwanathan, and Roman Garnett, 1–11. Long Beach: Curran Associates. https://​proceedings​.neurips​.cc​/paper​/2017​/hash​/3f5ee243547dee91fbd053c1c4a845aa​-Abstract​.html
Vieira, Lucas Nunes, and Elisa Alonso
2020 “Translating Perceptions and Managing Expectations: An Analysis of Management and Production Perspectives on Machine Translation.” Perspectives 28 (2): 163–184. DOI logoGoogle Scholar
Vygotsky, Lev S.
1978 Mind in Society: Development of Higher Psychological Processes. 14th ed. Harvard: Harvard University Press.Google Scholar