Cover not available
Article published In:
Computer-mediated communication in class: Fostering access to digital mediascapes
Edited by Stefania Maci and Marianna Lya Zummo
[Translation and Translanguaging in Multilingual Contexts 10:1] 2024
► pp. 28–53

Towards online audiovisual translation of videos

| University of Messina

The paper reports on the progress made by second-year language degree university students as regards their use of online corpus construction, annotation and search tools when exploring video genres (Baldry and Thibault 2010, 2020; Jablonkai and Csomay 2022). Through first-hand experiences undertaken with students using the OpenMWS platform (Taibi 2020; http://​mws​.pa​.itd​.cnr​.it/ ), the paper describes the ways in which, through the creation of new interactive communities, participation in the Messina OVP (Online Video Project) has empowered these students with regard to their acquisition of textual competences and, above all, their preparation for more intensive AVT studies in their third and final year. As such, the paper is a first-step towards online audiovisual translation that lays the bases for subsequent translation work and which is thus concerned with the underlying cohesion and coherence of the overall text as well as the analysis of specific textual formations in these videos.

Keywords: multimodality, AVT (audiovisual translation), cohesion, coherence, video corpora
Article outline
  • 1.Introduction
  • 2.Materials and methodology
    • 2.1Stage 1: Video selection
    • 2.2Stage 2: The transcript
    • 2.3Stage 3: Sequencing
    • 2.4Stage 4: Synchronising oral discourse-written discourse
    • 2.5Stage 5: Language analysis
    • 2.6Stage 6: Translation and subtitling
  • 3.Findings
  • 4.Discussion
  • 5.Conclusion
  • Notes
  • References
Published online: 9 February 2024
DOI logo
https://doi.org/10.1075/ttmc.00125.cam
References
Ackerley, Katherine, and Francesca Coccetta
2007 “Enriching Language Learning through a Multimedia Corpus.” ReCALL 19(3): 351–370. DOI logoGoogle Scholar
Aston, Guy, Silvia Bernardini, and Dominic Stewart
eds. 2004 Corpora and Language Learners. Amsterdam: John Benjamins. DOI logoGoogle Scholar
Baldry, Anthony
2021 “Multimodality and Genre Evolution. A Decade-by-decade Approach to Online Video Genre Analysis.” In Mediation and Multimodal Meaning Making in Digital Environments, ed. by Ilaria Moschini, and Maria Grazia Sindoni, 151–167. London: Routledge. DOI logoGoogle Scholar
Baldry, Anthony, and Paul Thibault
2001 “Towards Multimodal Corpora.” In Corpora in the Description and Teaching of English, ed. by Guy Aston, and Lou Burnard, 87–102. Bologna: CLUEB.Google Scholar
2006 “Multimodal Corpus Linguistics.” In System and Corpus: Exploring Connections, ed. by Geoff Thompson, and Susan Hunston, 164–183. London: Equinox.Google Scholar
2010 [2006] Multimodal Transcription and Text Analysis. A Multimedia Toolkit with Associated On-line Course. London: Equinox.Google Scholar
Baldry, Anthony, Alessandro Gaggia, and Marco Porta
2011 “Multimodal Web Concordancing and Annotation: An Overview of the MCAWEB System.” In Identities in Transition in the English-speaking World, ed. by Nicoletta Vasta, Antonella Riem Natale, Maria Bortoluzzi, and Debora Saidero, 39–60. Udine: Forum Editrice.Google Scholar
Baldry, Anthony, and Paul Thibault
2020 “Analysis A: A Model for Multimodal Corpus Construction of Video Genres: Phasal Analysis and Resource Integration Using OpenMWS.” In Multiliteracy Advances and Multimodal Challenges in ELT Environments, ed. by Nicoletta Vasta, and Anthony Baldry, 159–173. Udine: Forum Editrice.Google Scholar
Baldry, Anthony, Deirdre Kantz, Anna Loiacono, Ivana Marenzi, Davide Taibi, and Francesca Tursi
2020 “THE MWSWEB PROJECT: Accessing Medical Discourse in Video-hosting Websites.” Lingue e Linguaggi 401: 433–472.Google Scholar
Baldry, Anthony, and Deirdre Kantz
2022 “Corpus-assisted Approaches to Online Multimodal Discourse Analysis of Videos.” In Analyzing Multimodality in Specialized Discourse Settings. Innovative Research Methods and Applications, ed. by Veronica Bonsignori, Belinda Crawford Camiciottoli, and Denise Filmer, 1–22. Wilmington: Vernon Press.Google Scholar
Baldry, Anthony, Francesca Coccetta, and Deirdre Kantz
2022 “What if? Healthcare Simulations, Online Searchable Video Corpora and Formulating Hypotheses.” In Analysing Health Discourse in Digital Environments, ed. by Anna F. Plastina, 126–146. Newcastle upon Tyne: Cambridge Scholars Publishing.Google Scholar
Bateman, John
2017 “Triangulating Transmediality: A Multimodal Semiotic Framework Relating Media, Modes and Genres.” Discourse, Context Media 201: 160–174. DOI logoGoogle Scholar
Bianchi, Francesca, Davide Taibi, Philipp Kemkes, and Ivana Marenzi
2022 “Learning Analytics at the Service of Interpreter Training in Academic Curricula.” In Corpus Linguistics and Translation Tools for Digital Humanities, ed. by Stefania Maci, 153–176. London: Bloomsbury. DOI logoGoogle Scholar
Bogucki, Łukasz, and Jorge Díaz-Cintas
2020 “An Excursus on Audiovisual Translation.” In The Palgrave Handbook of Audiovisual Translation and Media Accessibility, ed. by Łukasz Bogucki, and Mikołaj Deckert, 11–32. London: Palgrave Macmillan. DOI logoGoogle Scholar
Böhme, Gernot
2012 Invasive Technification. Critical Essays in the Philosophy of Technology. London: Bloomsbury.Google Scholar
Bolter, David, and Richard Grusin
1998 Remediation. Understanding New Media. Boston: MIT Press.Google Scholar
Boulton, Alex
2017 “Data-Driven Learning and Language Pedagogy.” In Language, Education and Technology, ed. by Steven Thorne, and Stephen May, 181–192. New York: Springer. DOI logoGoogle Scholar
Boulton, Alex, and Nina Vyatkina
2021 “Thirty Years of Data-driven Learning: Taking Stock and Charting New Directions.” Language Learning & Technology 25(3): 66–89.Google Scholar
Brown, Gillian, and George Yule
1983 Discourse Analysis. Cambridge: Cambridge University Press. DOI logoGoogle Scholar
Bruti, Silvia
2018 “Spoken Discourse and Conversational Interaction in Audiovisual Translation.” In The Routledge Handbook of Audiovisual Translation, ed. by Luis Pérez-González, 192–208. London: Routledge. DOI logoGoogle Scholar
Cambria, Mariavita
2012 ““I’ll Never Say I’m sorry”: A Former Italian Prime Minister in a Corpus of English Online Newspapers.” Web Genres and Web Tools, ed. by Mariavita Cambria, Cristina Arizzi, and Francesca Coccetta, 277–294. Como: Ibis.Google Scholar
Cambria, Mariavita, Cristina Arizzi, and Francesca Coccetta
eds. 2012 Web Genres and Web Tools. Como: Ibis.Google Scholar
Chaume, Frederic
2018 “An Overview of Audiovisual Translation: Four Methodological Turns in a Mature Discipline.” Journal of Audiovisual Translation 1(1): 40–63. DOI logoGoogle Scholar
Coccetta, Francesca
2022 “Multimodal Corpora and Concordancing in Data-driven Learning.” In The Routledge Handbook of Corpora in English Language Teaching and Learning, ed. by Reka Jablonkai, and Eniko Csomay. London: Routledge. DOI logoGoogle Scholar
Cronin, Martin
2013 “Translation and Globalization.” In The Routledge Handbook of Translation Studies, ed. by Carmen Millán, and Francesca Bartrina, 491–502. London: Routledge.Google Scholar
De Beaugrande, Robert-Alain, and Wolfang Dressler
1981 Introduction to Text Linguistics. Harlow: Longman. DOI logoGoogle Scholar
Díaz-Cintas, Jorge
2018 “Subtitling’s a Carnival’: New Practices in Cyberspace.” The Journal of Specialised Translation 301: 127–149.Google Scholar
Díaz-Cintas, Jorge, and Serenella Massidda
2019 “Technological Advances in Audiovisual Translation.” In The Routledge Handbook of Translation and Technology, ed. by Minako O’Hagan, 255–270. London: Routledge. DOI logoGoogle Scholar
Dontcheva-Navratilova, Olga, and Renata Povolná
eds. 2009 Coherence and Cohesion in Spoken and Written Discourse. Newcastle upon Tyne: Cambridge Scholars Publishing.Google Scholar
Greco, Gian Maria
2018 “The Nature of Accessibility Studies.” Journal of Audiovisual Translation 1(1): 205–232. DOI logoGoogle Scholar
Halliday, Michael A. K., and Ruqaiya Hasan
1976 Cohesion in English. London: Arnold.Google Scholar
Hansen-Schirra, Silvia, Stella Neumann, and Eric Steiner
2007 “Cohesive Explicitness and Explicitation in an English-German Translation Corpus.” Language Contrast 7 (2): 241–265. DOI logoGoogle Scholar
Herring, Susan C.
2013 “Discourse in Web 2.0: Familiar, Reconfigured, and Emergent.” In Discourse 2.0: Language and New Media, ed. by Deborah Tannen, and Anne M. Trester, 1–25. Washington, DC: Georgetown University Press.Google Scholar
Hoffmann, Christian R.
2012 Cohesive Profiling: Meaning and Interaction in Personal Weblogs. Amsterdam: John Benjamins. DOI logoGoogle Scholar
Hunston, Susan, and Gill Francis
1995 Pattern Grammar. A Corpus-Driven Approach to the Lexical Grammar of English. Amsterdam: John Benjamins.DOI logoGoogle Scholar
Jablonkai, Reka, and Eniko Csomay
eds. 2022 The Routledge Handbook of Corpora in English Language Teaching and Learning. London: Routledge. DOI logoGoogle Scholar
Johns, Tim, and Florence Davies
1983 “Text as a Vehicle for Information: The Classroom Teaching of Reading in ESP.” Reading in a Foreign Language 1(1): 1–19.Google Scholar
Johns, Tim
1993 “Data-driven Learning: An Update.” TELL&CALL 21: 4–10.Google Scholar
1997 “Contexts: The Background, Development and Trialling of a Concordance-based CALL Program.” In Teaching and Language Corpora, ed. by Anne Wichmann, Steven Fligelstone, Tom McEnery, and Gerry Knowles, 100–115. Harlow: Longman.Google Scholar
2002 “Data-driven Learning: The Perpetual Challenge.” In Teaching and Learning by Doing Corpus Analysis, ed. by Bernhard Kettemann, and George Marko, 107–117. Amsterdam: Rodopi. DOI logoGoogle Scholar
Klein, Yvonne
2012 “Cohesion in English and German.” In Cross-Linguistic Corpora for the Study of Translations: Insights from the Language Pair English-German, ed. by Silvia Hansen-Schirra, Stella Neumann, and Eric Steiner, 161–172. Berlin: De Gruyter Mouton. DOI logoGoogle Scholar
Liu, Yu, and Kate O’Halloran
2009 “Intersemiotic Texture: Analyzing Cohesive Devices Between Language and Images.” Social Semiotics 19(4): 367–388. DOI logoGoogle Scholar
O’Hagan, Minako
2016 “Massively Open Translation: Unpacking the Relationship Between Technology and Translation in the 21st Century.” International Journal of Communication 101(2016): 929–946.Google Scholar
ed. 2019 The Routledge Handbook of Translation and Technology. London: Routledge. DOI logoGoogle Scholar
Pavesi, Maria, and Elisa Perego
2008 “Tailor-made Interlingual Subtitling as a Means to Enhance Second Language Acquisition.” In The Didactics of Audiovisual Translation, ed. by Jorge Díaz-Cintas, 215–225. Amsterdam: John Benjamins. DOI logoGoogle Scholar
Pavesi, Maria
2018 “Corpus-based Audiovisual Translation Studies: Ample Room for Development.” The Routledge Handbook of Audiovisual Translation, ed. by Luis Pérez-González, 315–333. London: Routledge. DOI logoGoogle Scholar
Perego, Elisa, and Christopher Taylor
eds. 2022 The Routledge Handbook of Audio Description. London: Routledge.Google Scholar
Sinclair, John
1991 Corpus, Concordance, Collocation. Oxford: Oxford University Press.Google Scholar
Sindoni, Maria Grazia
2013 Spoken and Written Discourse in Online Interactions: A Multimodal Approach. London: Routledge.Google Scholar
Sindoni, Maria Grazia, Elisabetta Adami, Styliani Karatza, Ivana Marenzi, Ilaria Moschini, and Sandra Petroni
2019 The Common Framework of Reference for Intercultural Digital Literacies. [URL]
Steiner, Erich
2017 “Contrastive Studies of Cohesion and their Impact on our Knowledge of Translation.” In Discourse Analysis in Translation Studies, ed. by Jeremy Munday, and Meifan Zhang, 27–45. Amsterdam: John Benjamins. DOI logoGoogle Scholar
Taibi, Davide
2020 “Analysis D: Shaping Digital Identities through Group Study: A Simulated Case Study Using Online Tools and Videos to Explore Animal-human Interactions.” In Multiliteracy Advances and Multimodal Challenges in ELT Environments, ed. by Nicoletta Vasta, and Anthony Baldry, 190–195. Udine: Forum Editrice.Google Scholar
Taibi, Davide, Deirdre Kantz, and Giovanni Fulantelli
2014 “Supporting Formative Assessment in Content and Language Integrated Learning: The MWS-Web Platform.” International Journal of Technology Enhanced Learning 6(4): 361–379. DOI logoGoogle Scholar
Taibi, Davide, Ivana Marenzi, Deirdre Kantz, and Giovanni Fulantelli
2015 “MWS-WEB: A System to Support a Pedagogical Approach Based on the Concept of Multiliteracies.” Italian Journal of Educational Technology 23(3): 164–171.Google Scholar
Taibi, Davide, Ivana Marenzi, and Qazi Asim Ijaz Ahmad
2019 “Ain’t That Sweet: Reflections on Scene Level Indexing and Annotation in the House Corpus Project.” In Representing and Redefining Specialised Knowledge: Medical Discourse, ed. by Anthony Baldry, Francesca Bianchi, and Anna Loiacono, 151–181. Lecce: ESE University of Salento Publishing.Google Scholar
Tanskanen, Sanna-Kaisa
2006 Collaborating towards Coherence: Lexical Cohesion in English Discourse. Amsterdam: John Benjamins. DOI logoGoogle Scholar
Taylor, Christopher
2003 “Multimodal Transcription in the Analysis: Translation and Subtitling of Italian Films.” The Translator 9(2): 191–206. DOI logoGoogle Scholar
2004 “Multimodal Text Analysis and Subtitling.” Perspectives on Multimodality, ed. by Eija Ventola, Cassily Charles, and Martin Kaltenbacher, 153–172. Amsterdam: John Benjamins. DOI logoGoogle Scholar