The multimodal translation workshop as a method of creative inquiry: Acousmatic sound, affective perception, and experiential literacy

Madeleine Campbell and Ricarda Vidal
University of Edinburgh | King’s College London

Abstract

This article investigates the role of affective perception in the development of translation and experiential literacy through the medium of a multimodal translation workshop held with twelve arts practitioners, academics, and translators. Both the rationale for the workshop format and the interpretation of workshop outputs draw on a transdisciplinary framework spanning theories of intermediality and multimodality, the study of acousmatic sound, acoustic atmospheres, and corporeal music/sound reception. Adopting a phenomenographic approach, we discuss the role of the body and the senses in communication and how the sensory exercises developed for our workshop can provide access to the prenoetic nature of perception from both a cognitive and affective standpoint. Recognizing the narrative quality of participants’ comments, a deductive approach was taken to analyze their translations and reflections through the lens of narrative modes of acousmatic music. The article concludes with pedagogical implications on the basis of participants’ reflections. Our findings support the use of a multimodal online translation workshop as both a research method to investigate meaning-making and a pedagogical resource to develop experiential literacy for both practitioners and developing translators.

Keywords:
Publication history

Date received: 30 March 2022

Date accepted: 31 January 2024

Published online: 1 March 2024

Table of contents

In 2010, Clive Scott published an article in a special issue of the art history journal Art in Translation in which he illustratively explains the practice of literary translation as “centrifugal practice.” Drawing on synaesthesia and pointing to the multisensory and multimodal qualities inherent to reading, he focuses on the reader/translator and their contribution to the text’s becoming. He describes translation as “a cross-sensory journey, a journey in which the lexical is allowed associatively to generate what sense-experience it wishes to” (Scott 2010, 162). He demonstrates a range of such sense experiences in a series of translations of Apollinaire’s poem “Le Voyageur” (1913), for which he employs various media including printed text, collage, photography, water colours, and enamels. Scott (2010) explodes the text by showing the sheer infinite possibilities in which it can be experienced and to illustrate his argument that

Full-text access is restricted to subscribers. Log in to obtain additional credentials. For subscription information see Subscription & Price. Direct PDF access to this article can be purchased through our e-platform.

References

Alves, Fabio, and Arnt Lykke Jakobsen
eds. 2021 The Routledge Handbook of Translation and Cognition. Abingdon: Routledge.Google Scholar
Andean, James
2016 “Narrative Modes in Acousmatic Music.” In Sound and Narrative, edited by James Andean, special issue of Organised Sound 21 (3): 192–203. DOI logoGoogle Scholar
Andrews, Molly, Corinne Squire, and Maria Tamboukou
eds. 2013 Doing Narrative Research. 2nd ed. London: SAGE. DOI logoGoogle Scholar
Aryani, Arash, Erin S. Isbilen, and Morten H. Christiansen
2020 “Affective Arousal Links Sound to Meaning.” Psychological Science 31 (8): 978–986. DOI logoGoogle Scholar
Atkinson, Dwight
2019 “Beyond the Brain: Intercorporeality and Co-Operative Action for SLA Studies.” The Modern Language Journal 103 (4): 724–738. DOI logoGoogle Scholar
Bailenson, Jeremy N.
2021 “Nonverbal Overload: A Theoretical Argument for the Causes of Zoom Fatigue.” Technology, Mind, and Behavior 2 (1). DOI logoGoogle Scholar
Berger, Christopher C., and H. Henrik Ehrsson
2017 “The Content of Imagined Sounds Changes Visual Motion Perception in the Cross-Bounce Illusion.” Scientific Reports 7 (1): 40123. DOI logoGoogle Scholar
Böhme, Gernot
2000 “Acoustic Atmospheres.” Soundscape: The Journal of Acoustic Ecology 1 (1): 14–18.Google Scholar
Bower, Matthew, and Shaun Gallagher
2016 “Bodily Affects as Prenoetic Elements in Enactive Perception.” Phenomenology and Mind (4): 108–131.Google Scholar
Campbell, Madeleine
2017 “Entre l’audible et l’inaudible: Intersemiotic Translation of Mohammed Dib’s Poetry.” In Language – Literature – the Arts: A Cognitive-Semiotic Interface, edited by Elżbieta Chrzanowska-Kluczewska and Olga Voroboya, 167–182. Frankfurt: Peter Lang.Google Scholar
Campbell, Madeleine, Birthe Jørgensen, and Bethan. R. Parkes
eds. 2013 Haجar and the Anجel: A Response to John Runciman’s Painting ‘Hagar and the Angel’ (c1766). Collaborative Installation at The Hunterian, Glasgow, UK. https://​www​.gla​.ac​.uk​/hunterian​/learning​/hunterianassociates​/hagarinstallation/
Campbell, Madeleine, and Ricarda Vidal
eds. 2019 Translating across Sensory and Linguistic Borders: Intersemiotic Journeys between Media. Cham: Palgrave Macmillan. DOI logoGoogle Scholar
Creswell, John W., and J. David Creswell
2018 Research Design: Qualitative, Quantitative & Mixed Methods Approaches. Los Angeles: SAGE.Google Scholar
Damasio, Antonio. R.
1994 Descartes’ Error: Emotion, Reason, and the Human Brain. New York: Avon.Google Scholar
1999 The Feeling of What Happens: Body and Emotion in the Making of Consciousness. New York: Harcourt Brace.Google Scholar
Dib, Mohammed
(1996) 2007 “L’Aube Ismaël (louange) [Dawn Ismaël (Hymn of praise)].” In Oeuvres Complètes: I Poésies [Complete works: I Poems], edited by Habib Tengour. Paris: Éditions de la Différence.Google Scholar
Elleström, Lars
2010 “The Modalities of Media: A Model for Understanding Intermedial Relations.” In Media Borders, Multimodality and Intermediality, edited by Lars Elleström, 11–48. Basingstoke: Palgrave Macmillan. DOI logoGoogle Scholar
2016 “Visual Iconicity in Poetry: Replacing the Notion of ‘Visual Poetry’.” Orbis Litterarum 71 (6): 437–472. DOI logoGoogle Scholar
Englund, Axel
2010 “Intermedial Topography and Metaphorical Interaction.” In Media Borders, Intermediality and Multimodality, edited by Lars Elleström, 69–80. Basingstoke: Palgrave Macmillan. DOI logoGoogle Scholar
Erickson, Frederick
2011 “Reflections on Practice, Teaching/Learning, Video, and Theorizing.” In Theories of Learning and Studies of Instructional Practice, edited by Timothy Koschmann, 385–402. New York: Routledge. DOI logoGoogle Scholar
ffrench, Patrick
2015 “Barthes and the Voice: The Acousmatic and Beyond.” L’Esprit créateur 55 (4): 56–69. DOI logoGoogle Scholar
Gibson, James J.
1979 The Ecological Approach to Visual Perception. Boston: Houghton Mifflin.Google Scholar
Greener, Ian
2011 Designing Social Research: A Guide for the Bewildered. London: SAGE. DOI logoGoogle Scholar
Hajar, Anas
2021 “Theoretical Foundations of Phenomenography: A Critical Review.” Higher Education Research and Development 40 (7): 1421–1436. DOI logoGoogle Scholar
Halliday, M. A. K.
1978 Language as Social Semiotic: The Social Interpretation of Language and Meaning. London: Hodder Arnold.Google Scholar
Hokkanen, Sari, and Kaisa Koskinen
2016 “Affect as a Hinge: The Translator’s Experiencing Self as a Sociocognitive Interface.” Translation Spaces 5 (1): 78–96.Google Scholar
Hutchins, Edwin
1995 Cognition in the Wild. Cambridge: MIT Press.Google Scholar
Katschthaler, Karl
2022 Zwischen Atmosphäre und Narration: Zum Verhältnis von Musik, Sprache und Literatur im 20. und 21. Jahrhundert [Between atmosphere and narration: The relationship of music, language and literature in the 20th and 21st century]. Bielefeld: Transcript Verlag.Google Scholar
Kellough, B. R.
2013 “Sound Installation.” In Haجar and the Anجel: A Response to John Runciman’s Painting ‘Hagar and the Angel’ (c1766), edited by Madeleine Campbell, Birthe Jørgensen, and Bethan. R. Parkes. Collaborative Installation at The Hunterian, Glasgow, UK. https://​www​.gla​.ac​.uk​/hunterian​/learning​/hunterianassociates​/hagarinstallation/
Kohler Riessman, Catherine
2008 Narrative Methods for the Human Sciences. Thousand Oaks: SAGE.Google Scholar
Koskinen, Kaisa
2012 “Domestication, Foreignization and the Modulation of Affect.” In Domestication and Foreignization in Translation Studies, edited by Hannu Kemppainen, Marja Jänis, and Alexandra Belikova, 13–32. Berlin: Frank & Timme.Google Scholar
2015 “Training Translators for a Superdiverse World: Translators’ Intercultural Competence and Translation as Affective Work.” Russian Journal of Linguistics 19 (4): 175–184.Google Scholar
2020 Translation and Affect: Essays on Sticky Affects and Translational Affective Labour. Amsterdam: John Benjamins. DOI logoGoogle Scholar
Kress, Gunther R.
2003 Literacy in the New Media Age. London: Routledge.Google Scholar
2010 Multimodality: A Social Semiotic Approach to Contemporary Communication. Abingdon: Routledge.Google Scholar
Kress, Gunther R., and Theo van Leeuwen
1996 Reading Images: The Grammar of Visual Design. Abingdon: Routledge.Google Scholar
Kress, Gunther R., and Theo Van Leeuwen
2021 Reading Images: The Grammar of Visual Design. 3rd ed. Abingdon: Routledge.Google Scholar
Leavy, Patricia
2017 Research Design: Quantitative, Qualitative, Mixed Methods, Arts-Based, and Community-Based Participatory Research Approaches. New York: The Guilford PressGoogle Scholar
Lebrecht, Sophie, Moshe Bar, Lisa Feldman Barrett, and Michael J. Tarr
2012 “Micro-Valences: Perceiving Affective Valence in Everyday Objects.” Frontiers in Psychology 3: 107. DOI logoGoogle Scholar
Lehr, Caroline
2021 “Translation, Emotion and Cognition.” In Alves and Jakobsen (2021, 294–309). DOI logoGoogle Scholar
Leibowitz, Brenda
2009 “The Professional Development of Academics as Teachers.” In Researching Higher Education: International Perspectives on Theory, Policy and Practice, edited by Jennifer M. Case and Jeroen Huisman, 153–170. New York: Routledge.Google Scholar
Marton, Ference
1986 “Phenomenography: A Research Approach to Investigating Different Understandings of Reality.” Journal of Thought 21 (3): 28–49.Google Scholar
Marton, Ference, and Shirley Booth
1997 Learning and Awareness. Mahwah: Lawrence Erlbaum.Google Scholar
Martin, J. R., and White, R.
2005 The Language of Evaluation: Appraisal in English. Basingstoke: Palgrave Macmillan. DOI logoGoogle Scholar
Massey, Doreen
2005 For Space. London: Sage.Google Scholar
Meschonnic, Henri
2007 Éthique et politique du traduire [The ethics and politics of translation]. Paris: Verdier.Google Scholar
Meyer, Christian, Jürgen Streeck, and J. Scott Jordan
eds. 2017 Intercorporeality: Emerging Socialities in Interaction. Oxford: Oxford University Press. DOI logoGoogle Scholar
Muñoz Martín, Ricardo
2016 “Processes of What Models? On the Cognitive Indivisibility of Translation Acts and Events.” In Cognitive Space: Exploring the Situational Interface, edited by Birgitta Englund Dimitrova and Maureen Ehrensberger-Dow, special issue of Translation Spaces 5 (1): 145–161.Google Scholar
Muñoz Martín, Ricardo, and Celia Martín de León
2021 “Translation and Cognition.” In Alves and Jakobsen (2021, 52–68). DOI logoGoogle Scholar
Noë, Alva
2004 Action in Perception. Cambridge, MA: MIT Press.Google Scholar
O’Regan, J. Kevin, and Alve Noë
2001a “A Sensorimotor Account of Vision and Visual Consciousness.” Behavioral and Brain Sciences 24 (5): 939–973. DOI logoGoogle Scholar
2001b “What Is It Like to See: A Sensorimotor Theory of Perceptual Experience.” Synthese 129 (1): 79–103. DOI logoGoogle Scholar
Painter, Clare, J. R. Martin, and Len Unsworth
2013 Reading Visual Narratives: Image Analysis of Children’s Picture Books. Sheffield: Equinox Publishing.Google Scholar
Petrides, K. V.
2009 Technical Manual for the Trait Emotional Intelligence Questionnaires (TEIQue). London: London Psychometric Laboratory.Google Scholar
Pherali, Tejendra J.
2011 Phenomenography as a Research Strategy: Researching Environmental Conceptions. Saarbrücken: Lambert Academic Publishing.Google Scholar
Raguet, Christine
2016 “Alienability and Creativity: The Role of Sounds and Sensations in Translation.” In Translation and Creativity – La traduction comme création, edited by Martine Hennard Dutheil de la Rochère and Irene Weber-Henking, 135–160. Lausanne: Cahiers du CTL.Google Scholar
Risku, Hanna, and Regina Rogl
2021 “Translation and Situated, Embodied, Distributed and Extended Cognition.” In Alves and Jakobsen (2021, 478–499). DOI logoGoogle Scholar
Rowell, Lonnie L., Margaret M. Riel, and Elena Yu. Polush
2017 “Defining Action Research: On Dialogic Spaces for Constructing Shared Meanings.” In The Palgrave International Handbook of Action Research, edited by Lonnie L. Rowell, Catherine D. Bruce, Joseph M. Shosh, and Margaret M. Riel, 85–101. New York: Palgrave Macmillan. DOI logoGoogle Scholar
Rudenko, Svetlana, and Maria José de Córdoba Serrano
2017 “Musical-Space Synaesthesia: Visualisation of Musical Texture.” Multisensory Research 30 (3–5): 279–285. DOI logoGoogle Scholar
Schaeffer, Pierre
1966 Traité des objets musicaux [A treatise on musical objects]. Paris: Le Seuil.Google Scholar
Scott, Clive
2010 “Intermediality and Synesthesia: Literary Translation as Centrifugal Practice.” Art in Translation 2 (2): 153–169. DOI logoGoogle Scholar
Sheets-Johnstone, Maxine
2011 “Embodied Minds or Mindful Bodies? A Question of Fundamental, Inherently Inter-related Aspects of Animation.” Subjectivity 4 (4): 451–466. DOI logoGoogle Scholar
Silverstein, Michael
2015 “How Language Communities Intersect: Is ‘Superdiversity’ an Incremental or Transformative Condition?” Language and Communication 44: 7–18. DOI logoGoogle Scholar
Stuart, Susan A. J.
2017 “Feeling Our Way: Enkinesthetic Enquiry and Immanent Intercorporeality.” In Intercorporeality: Emerging Socialities in Interaction, edited by Christian Meyer, Jürgen Streeck, and J. Scott Jordan, 51–72. Oxford: Oxford University Press.Google Scholar
Van Lier, Leo
2008 “Ecological-Semiotic Perspectives on Educational Linguistics.” In The Handbook of Educational Linguistics, edited by Bernard Spolsky and Francis M. Hult, 596–605. Malden, MA: Blackwell. DOI logoGoogle Scholar
Vatakis, Argiro, and Charles Spence
2007 “Crossmodal Binding: Evaluating the ‘Unity Assumption’ Using Audiovisual Speech Stimuli.” Perception & Psychophysics 69 (5): 744–756. DOI logoGoogle Scholar
Venuti, Lawrence
1995 The Translator’s Invisibility: A History of Translation. Abingdon: Routledge.Google Scholar
Ward, Jamie
2008 The Frog Who Croaked Blue: Synesthesia and the Mixing of the Senses. Hove: Routledge.Google Scholar
Williams, Ralph Vaughan
1925 “The Lark Ascending: Romance for Violin and Orchestra.” Score. Oxford: Oxford University Press.Google Scholar