Cover not available
Article In:
Approaches to Machine Translation
Edited by Mahdieh Fakhar, Monica Vilhelm and Paz Díez-Arcón
[Translation and Translanguaging in Multilingual Contexts 11:1] 2025
► pp. 48–63

Machine translation of tourism reviews

Quality assessment and localization

| Universitat Politècnica de València
| Universitat Politècnica de València
Machine translation (MT) has surpassed all quality expectations and its use has increased exponentially in recent years (Forcada 2017; Sánchez-Gijón, Moorkens, and Way 2019). One of the most frequent MT applications is the translation of user-generated content (UGC) and, more specifically, reviews on tourism portals such as Tripadvisor. Several authors agree that the degree of trust and credibility of a review, as the most important characteristics of UGC, depends largely on the perceived naturalness and authenticity of its writing (Pollach 2006; Schemmann 2011; Vásquez 2014). The review’s influence on the product’s reputation and on the purchase decision-making of future users has been fully demonstrated. Since review platforms make reviews available to users in different languages translated by MT, the quality of MT output should be studied from the point of view of the text’s adaptation to the requirements of a specific audience and market, following the principles elaborated in localization studies. The aim of this paper is to analyze the behavior of neural MT of user-generated content from the perspective of localization to check whether MT quality depends exclusively on linguistic or stylistic aspects or whether the aspects studied by localization, such as linguistic and cultural appropriateness for the target user, also play a decisive role. We compiled an English-Spanish parallel corpus consisting of 250 reviews retrieved from Tripadvisor. The reviews were originally written in English and MT processed into Spanish. Then the quality of the MT output was evaluated following two parameters: correctness and acceptance according to MT quality scales and localization guidelines.
Keywords: localization, machine translation, user generated content, quality
Article outline
  • 1.Introduction
  • 2.Importance and characteristics of UGC
  • 3.Current status of MT: Characteristics of neural machine translation
    • 3.1Quality parameters and metrics
    • 3.2Machine translation and localization
  • 4.Methodology
  • 5.Analysis and discussion
    • 5.1Evaluation of the MT quality of reviews
    • 5.2Localization problems
      • 5.2.1Localization errors: Cultural variants
      • 5.2.2Localization errors: Linguistic variant
    • 5.3Other causes affecting quality
      • 5.3.1Errors caused by explanations or duplications in the original text
      • 5.3.2Errors caused by typos in the original text
    • 5.4Summary of results
  • 6.Conclusions
  • Author queries
  • References
This content is being prepared for publication; it may be subject to changes.
References (30)
References
Arevalillo Doval, Juan J. 2000. “La Localización: concepto, nuevas tecnologías y requisitos.” In Traducción y nuevas tecnologías, herramientas auxiliares del traductor: encuentros en torno a la traducción, ed. by Carmen Valero Garcés, and Isabel de la Cruz Cabanillas, 115–124. Alcalá de Henares: Universidad de Alcalá de Henares.Google Scholar
Arivazhagan, Naveen, Ankur Bapna, Orhan Firat, Roee Aharoni, Melvin Johnson, and Wolfgang Macherey. 2019. “The Missing Ingredient in Zero-Shot Neural Machine Translation.” arXiv. [URL]
Bahdanau, Dzmitry, Kyunghyun Cho, and Yoshua Bengio. 2016. “Neural Machine Translation by Jointly Learning to Align and Translate.” arXiv. [URL]
Banerjee, Snehasish, and Alton Y. K. Chua. 2014. “A Linguistic Framework to Distinguish Between Genuine and Deceptive Online Reviews.” In Proceedings of The International MultiConference of Engineers and Computer Scientists 2014, ed. by Sio Iong Ao, Oscar Castillo, Craig Douglas, David Dagan Feng, and Jeong-A. Lee, 501–506. Hong Kong: International Association of Engineers. [URL]
Cadieux, Pierre, and Bert Esselink. 2004. “GILT: Globalization, Internationalization, Localization, Translation.” Globalization Insider 11(1): 1–5.Google Scholar
Candel-Mora, Miguel A. 2015. “Evaluation of English to Spanish MT Output of Tourism 2.0 Consumer-Generated Reviews with Post-Editing Purposes.” In Proceedings of the 37th Conference Translating and the Computer, ed. by Joao Esteves-Ferreira, Juliet Macan, Ruslan Mitkov, and Olaf-Michael Stefanov, 37–47. London: AsLing. [URL]
Castilho, Sheila, Joss Moorkens, Federico Gaspari, Iacer Calixto, John Tinsley, and Andy Way. 2017. “Is Neural Machine Translation the New State of the Art?.” The Prague Bulletin of Mathematical Linguistics 108 (1): 109–120. DOI logoGoogle Scholar
Doherty, Stephen. 2017. “Issues in Human and Automatic Translation Quality Assessment.” In Human Issues in Translation Technology, ed. by Dorothy Kenny, 131–148. London: Routledge.Google Scholar
Esselink, Bert. 2003. “The Evolution of Localization.” Multilingual Computing and Technology 14 (5): 4–7.Google Scholar
Fiederer, Rebecca, and Sharon O’Brien. 2009. “Quality and Machine Translation: A Realistic Objective.” The Journal of Specialised Translation 11(1): 52–74.Google Scholar
Forcada, Mikel L. 2017. “Making Sense of Neural Machine Translation.” Translation Spaces 6(2): 291–309. DOI logoGoogle Scholar
Görög, Attila. 2014. “Quality Evaluation Today: The Dynamic Quality Framework.” In Proceedings of Translating and the Computer 36, ed. by João Esteves-Ferreira, Juliet Macan, Ruslan Mitkov, and Olaf-Michael Stefanov, 155–164. London: AsLing. [URL]
Jean, Sébastien, Kyunghyun Cho, Roland Memisevic, and Yoshua Bengio. 2015. “On Using Very Large Target Vocabulary for Neural Machine Translation.” In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing of the Asian Federation of Natural Language Processing, Volume 1: Long Papers, ed. by Chengqing Zong, and Michael Strube, 1–10. Beijing: Association for Computational Linguistics. DOI logoGoogle Scholar
Lommel, Arle, Hans Uszkoreit, and Aljoscha Burchardt. 2014. “Multidimensional Quality Metrics (MQM): A Framework for Declaring and Describing Translation Quality Metrics.” Revista Tradumàtica 121: 455–463. DOI logoGoogle Scholar
O’Brien, Sharon, Rahzeb Choudhury, Jaap van der Meer, and Nora Aranberri. 2011. TAUS Dynamic Quality Evaluation Framework: TAUS Labs report. [URL]
O’Hagan, Minako, and David Ashworth. 2002. Translation-Mediated Communication in a Digital World. Facing the Challenges of Globalization and Localization. Clevedon: Multilingual Matters.Google Scholar
Pagans, Marta. 2002. “Localització, ens ubiquem?.” Revista Tradumàtica 11: 1–4.Google Scholar
Pérez Fernández, Lucila. 2010. La localización de videojuegos (inglés-español): aspectos técnicos, metodológicos y profesionales. PhD diss. Universidad de Málaga.
Peris-Abril, Álvaro, Miguel Domingo, and Francisco Casacuberta. 2017. “Interactive Neural Machine Translation.” Computer Speech & Language 451: 201–220. DOI logoGoogle Scholar
Pollach, Irene. 2006. “Electronic Word of Mouth: A Genre Analysis of Product Reviews on Consumer Opinion Web Sites.” In Proceedings of the 39th Hawaii International Conference on System Sciences, ed. by Ralph Sprague Jr., Vol. 31, 1530–1605. Washington D.C.: IEEE Computer Society. [URL]. DOI logo
Pym, Anthony. 2012. Teorías contemporáneas de la traducción (exploraciones pedagógicas). Tarragona: Intercultural Studies Group- URV.Google Scholar
Ricci, Francesco, and René Wietsma. 2006. “Product Reviews in Travel Decision-Making.” In Information and Communication Technologies in Tourism, ed. by Martin Hitz, Marianna Sigala, and Jamie Murphy, 296–307. Vienna: Springer.Google Scholar
Sánchez-Gijón, Pilar, Joss Moorkens, and Andy Way. 2019. “Post-Editing Neural Machine Translation Versus Translation Memory Segments.” Machine Translation 33(2): 31–59. DOI logoGoogle Scholar
Schemmann, Brita. 2011. “A Classification of Presentation Forms of Travel and Tourism-Related Online Consumer Reviews.” e-Review of Tourism Research 9 (2): 14–24. [URL]
Scholand, Michael. 2002. “Localización de Videojuegos.” Revista Tradumàtica 11: 1–9.Google Scholar
Schuster, Mike, Melvin Johnson, and Nikhil Thorat. 2016. “Zero-Shot Translation with Google’s Multilingual Neural Machine Translation System.” Google AI Blog 22 1. [URL]
Tripadvisor. Accessed March 10, 2024. [URL]
Valli, Paola. 2015. “The TAUS Quality Dashboard.” In Proceedings of the 37th Conference Translating and the Computer, ed. by Joao Esteves-Ferreira, Juliet Macan, Ruslan Mitkov, and Olaf-Michael Stefanov, 127–136. London: AsLing. [URL]
Vásquez, Camilla. 2012. “Narrativity and Involvement in Online Consumer Reviews: The Case of Tripadvisor.” Narrative Inquiry 22(1): 105–121. DOI logoGoogle Scholar
. 2014. ““Usually Not One to Complain But…”: Constructing Identities in User-Generated Online Reviews.” In The Language of Social Media, ed. by Caroline Tagg, and Philip Seargeant, 65–90. London: Palgrave Macmillan. DOI logoGoogle Scholar