Cover not available
Article published In:
Target
Vol. 36:2 (2024) ► pp.184–214

The multimodal translation workshop as a method of creative inquiry

Acousmatic sound, affective perception, and experiential literacy

ORCID logo | University of Edinburgh
ORCID logo | King’s College London
This article investigates the role of affective perception in the development of translation and experiential literacy through the medium of a multimodal translation workshop held with twelve arts practitioners, academics, and translators. Both the rationale for the workshop format and the interpretation of workshop outputs draw on a transdisciplinary framework spanning theories of intermediality and multimodality, the study of acousmatic sound, acoustic atmospheres, and corporeal music/sound reception. Adopting a phenomenographic approach, we discuss the role of the body and the senses in communication and how the sensory exercises developed for our workshop can provide access to the prenoetic nature of perception from both a cognitive and affective standpoint. Recognizing the narrative quality of participants’ comments, a deductive approach was taken to analyze their translations and reflections through the lens of narrative modes of acousmatic music. The article concludes with pedagogical implications on the basis of participants’ reflections. Our findings support the use of a multimodal online translation workshop as both a research method to investigate meaning-making and a pedagogical resource to develop experiential literacy for both practitioners and developing translators.
Keywords: experiential literacy, experiential translation, embodiment, cognition and affect, multimodality, intermediality
Article outline
  • 1.Introduction: The creative translator and the multisensorial experience of the text
  • 2.Modalities, affordances, affect, and the body
    • 2.1Language and embodied perception
    • 2.2Towards an affective phenomenology of (intersemiotic) translation
    • 2.3Sound, body, and language
    • 2.4Acousmatic listening and dépaysement
  • 3.Workshop and data generation
    • 3.1Researcher positionality
    • 3.2Research approach
  • 4.Analysis and discussion
    • 4.1Cognitive and affective analysis of listening and drawing exercises
    • 4.2Narrative analysis
      • 4.2.1Spatial narrative
      • 4.2.2Mimetic narrative
      • 4.2.3Material narrative
      • 4.2.4Embodied narrative
  • 5.Conclusion and pedagogical implications
  • Acknowledgements
  • Notes
  • References
Published online: 1 March 2024
DOI logo
https://doi.org/10.1075/target.22051.cam
References (69)
References
Alves, Fabio, and Arnt Lykke Jakobsen, eds. 2021. The Routledge Handbook of Translation and Cognition. Abingdon: Routledge.Google Scholar
Andean, James. 2016. “Narrative Modes in Acousmatic Music.” In Sound and Narrative, edited by James Andean, special issue of Organised Sound 21 (3): 192–203. DOI logoGoogle Scholar
Andrews, Molly, Corinne Squire, and Maria Tamboukou, eds. 2013. Doing Narrative Research. 2nd ed. London: SAGE. DOI logoGoogle Scholar
Aryani, Arash, Erin S. Isbilen, and Morten H. Christiansen. 2020. “Affective Arousal Links Sound to Meaning.” Psychological Science 31 (8): 978–986. DOI logoGoogle Scholar
Atkinson, Dwight. 2019. “Beyond the Brain: Intercorporeality and Co-Operative Action for SLA Studies.” The Modern Language Journal 103 (4): 724–738. DOI logoGoogle Scholar
Bailenson, Jeremy N. 2021. “Nonverbal Overload: A Theoretical Argument for the Causes of Zoom Fatigue.” Technology, Mind, and Behavior 2 (1). DOI logoGoogle Scholar
Berger, Christopher C., and H. Henrik Ehrsson. 2017. “The Content of Imagined Sounds Changes Visual Motion Perception in the Cross-Bounce Illusion.” Scientific Reports 7 (1): 40123. DOI logoGoogle Scholar
Böhme, Gernot. 2000. “Acoustic Atmospheres.” Soundscape: The Journal of Acoustic Ecology 1 (1): 14–18.Google Scholar
Bower, Matthew, and Shaun Gallagher. 2016. “Bodily Affects as Prenoetic Elements in Enactive Perception.” Phenomenology and Mind (4): 108–131.Google Scholar
Campbell, Madeleine. 2017. “Entre l’audible et l’inaudible: Intersemiotic Translation of Mohammed Dib’s Poetry.” In Language – Literature – the Arts: A Cognitive-Semiotic Interface, edited by Elżbieta Chrzanowska-Kluczewska and Olga Voroboya, 167–182. Frankfurt: Peter Lang.Google Scholar
Campbell, Madeleine, Birthe Jørgensen, and Bethan. R. Parkes, eds. 2013. Haجar and the Anجel: A Response to John Runciman’s Painting ‘Hagar and the Angel’ (c1766). Collaborative Installation at The Hunterian, Glasgow, UK. [URL]
Campbell, Madeleine, and Ricarda Vidal, eds. 2019. Translating across Sensory and Linguistic Borders: Intersemiotic Journeys between Media. Cham: Palgrave Macmillan. DOI logoGoogle Scholar
Creswell, John W., and J. David Creswell. 2018. Research Design: Qualitative, Quantitative & Mixed Methods Approaches. Los Angeles: SAGE.Google Scholar
Damasio, Antonio. R. 1994. Descartes’ Error: Emotion, Reason, and the Human Brain. New York: Avon.Google Scholar
1999. The Feeling of What Happens: Body and Emotion in the Making of Consciousness. New York: Harcourt Brace.Google Scholar
Dib, Mohammed. (1996) 2007. “L’Aube Ismaël (louange) [Dawn Ismaël (Hymn of praise)].” In Oeuvres Complètes: I Poésies [Complete works: I Poems], edited by Habib Tengour. Paris: Éditions de la Différence.Google Scholar
Elleström, Lars. 2010. “The Modalities of Media: A Model for Understanding Intermedial Relations.” In Media Borders, Multimodality and Intermediality, edited by Lars Elleström, 11–48. Basingstoke: Palgrave Macmillan. DOI logoGoogle Scholar
. 2016. “Visual Iconicity in Poetry: Replacing the Notion of ‘Visual Poetry’.” Orbis Litterarum 71 (6): 437–472. DOI logoGoogle Scholar
Englund, Axel. 2010. “Intermedial Topography and Metaphorical Interaction.” In Media Borders, Intermediality and Multimodality, edited by Lars Elleström, 69–80. Basingstoke: Palgrave Macmillan. DOI logoGoogle Scholar
Erickson, Frederick. 2011. “Reflections on Practice, Teaching/Learning, Video, and Theorizing.” In Theories of Learning and Studies of Instructional Practice, edited by Timothy Koschmann, 385–402. New York: Routledge. DOI logoGoogle Scholar
ffrench, Patrick. 2015. “Barthes and the Voice: The Acousmatic and Beyond.” L’Esprit créateur 55 (4): 56–69. DOI logoGoogle Scholar
Gibson, James J. 1979. The Ecological Approach to Visual Perception. Boston: Houghton Mifflin.Google Scholar
Greener, Ian. 2011. Designing Social Research: A Guide for the Bewildered. London: SAGE. DOI logoGoogle Scholar
Hajar, Anas. 2021. “Theoretical Foundations of Phenomenography: A Critical Review.” Higher Education Research and Development 40 (7): 1421–1436. DOI logoGoogle Scholar
Halliday, M. A. K. 1978. Language as Social Semiotic: The Social Interpretation of Language and Meaning. London: Hodder Arnold.Google Scholar
Hokkanen, Sari, and Kaisa Koskinen. 2016. “Affect as a Hinge: The Translator’s Experiencing Self as a Sociocognitive Interface.” Translation Spaces 5 (1): 78–96.Google Scholar
Hutchins, Edwin. 1995. Cognition in the Wild. Cambridge: MIT Press.Google Scholar
Katschthaler, Karl. 2022. Zwischen Atmosphäre und Narration: Zum Verhältnis von Musik, Sprache und Literatur im 20. und 21. Jahrhundert [Between atmosphere and narration: The relationship of music, language and literature in the 20th and 21st century]. Bielefeld: Transcript Verlag.Google Scholar
Kellough, B. R. 2013. “Sound Installation.” In Haجar and the Anجel: A Response to John Runciman’s Painting ‘Hagar and the Angel’ (c1766), edited by Madeleine Campbell, Birthe Jørgensen, and Bethan. R. Parkes. Collaborative Installation at The Hunterian, Glasgow, UK. [URL]
Kohler Riessman, Catherine. 2008. Narrative Methods for the Human Sciences. Thousand Oaks: SAGE.Google Scholar
Koskinen, Kaisa. 2012. “Domestication, Foreignization and the Modulation of Affect.” In Domestication and Foreignization in Translation Studies, edited by Hannu Kemppainen, Marja Jänis, and Alexandra Belikova, 13–32. Berlin: Frank & Timme.Google Scholar
. 2015. “Training Translators for a Superdiverse World: Translators’ Intercultural Competence and Translation as Affective Work.” Russian Journal of Linguistics 19 (4): 175–184.Google Scholar
Kress, Gunther R. 2003. Literacy in the New Media Age. London: Routledge.Google Scholar
2010. Multimodality: A Social Semiotic Approach to Contemporary Communication. Abingdon: Routledge.Google Scholar
Kress, Gunther R., and Theo van Leeuwen. 1996. Reading Images: The Grammar of Visual Design. Abingdon: Routledge.Google Scholar
Kress, Gunther R., and Theo Van Leeuwen. 2021. Reading Images: The Grammar of Visual Design. 3rd ed. Abingdon: Routledge.Google Scholar
Leavy, Patricia. 2017. Research Design: Quantitative, Qualitative, Mixed Methods, Arts-Based, and Community-Based Participatory Research Approaches. New York: The Guilford PressGoogle Scholar
Lebrecht, Sophie, Moshe Bar, Lisa Feldman Barrett, and Michael J. Tarr. 2012. “Micro-Valences: Perceiving Affective Valence in Everyday Objects.” Frontiers in Psychology 31: 107. DOI logoGoogle Scholar
Lehr, Caroline. 2021. “Translation, Emotion and Cognition.” In Alves and Jakobsen (2021, 294–309). DOI logoGoogle Scholar
Leibowitz, Brenda. 2009. “The Professional Development of Academics as Teachers.” In Researching Higher Education: International Perspectives on Theory, Policy and Practice, edited by Jennifer M. Case and Jeroen Huisman, 153–170. New York: Routledge.Google Scholar
Marton, Ference. 1986. “Phenomenography: A Research Approach to Investigating Different Understandings of Reality.” Journal of Thought 21 (3): 28–49.Google Scholar
Marton, Ference, and Shirley Booth. 1997. Learning and Awareness. Mahwah: Lawrence Erlbaum.Google Scholar
Martin, J. R., and White, R. 2005. The Language of Evaluation: Appraisal in English. Basingstoke: Palgrave Macmillan. DOI logoGoogle Scholar
Massey, Doreen. 2005. For Space. London: Sage.Google Scholar
Meschonnic, Henri. 2007. Éthique et politique du traduire [The ethics and politics of translation]. Paris: Verdier.Google Scholar
Meyer, Christian, Jürgen Streeck, and J. Scott Jordan, eds. 2017. Intercorporeality: Emerging Socialities in Interaction. Oxford: Oxford University Press. DOI logoGoogle Scholar
Muñoz Martín, Ricardo. 2016. “Processes of What Models? On the Cognitive Indivisibility of Translation Acts and Events.” In Cognitive Space: Exploring the Situational Interface, edited by Birgitta Englund Dimitrova and Maureen Ehrensberger-Dow, special issue of Translation Spaces 5 (1): 145–161.Google Scholar
Muñoz Martín, Ricardo, and Celia Martín de León. 2021. “Translation and Cognition.” In Alves and Jakobsen (2021, 52–68). DOI logoGoogle Scholar
Noë, Alva. 2004. Action in Perception. Cambridge, MA: MIT Press.Google Scholar
O’Regan, J. Kevin, and Alve Noë. 2001a. “A Sensorimotor Account of Vision and Visual Consciousness.” Behavioral and Brain Sciences 24 (5): 939–973. DOI logoGoogle Scholar
. 2001b. “What Is It Like to See: A Sensorimotor Theory of Perceptual Experience.” Synthese 129 (1): 79–103. DOI logoGoogle Scholar
Painter, Clare, J. R. Martin, and Len Unsworth. 2013. Reading Visual Narratives: Image Analysis of Children’s Picture Books. Sheffield: Equinox Publishing.Google Scholar
Petrides, K. V. 2009. Technical Manual for the Trait Emotional Intelligence Questionnaires (TEIQue). London: London Psychometric Laboratory.Google Scholar
Pherali, Tejendra J. 2011. Phenomenography as a Research Strategy: Researching Environmental Conceptions. Saarbrücken: Lambert Academic Publishing.Google Scholar
Raguet, Christine. 2016. “Alienability and Creativity: The Role of Sounds and Sensations in Translation.” In Translation and Creativity – La traduction comme création, edited by Martine Hennard Dutheil de la Rochère and Irene Weber-Henking, 135–160. Lausanne: Cahiers du CTL.Google Scholar
Risku, Hanna, and Regina Rogl. 2021. “Translation and Situated, Embodied, Distributed and Extended Cognition.” In Alves and Jakobsen (2021, 478–499). DOI logoGoogle Scholar
Rowell, Lonnie L., Margaret M. Riel, and Elena Yu. Polush. 2017. “Defining Action Research: On Dialogic Spaces for Constructing Shared Meanings.” In The Palgrave International Handbook of Action Research, edited by Lonnie L. Rowell, Catherine D. Bruce, Joseph M. Shosh, and Margaret M. Riel, 85–101. New York: Palgrave Macmillan. DOI logoGoogle Scholar
Rudenko, Svetlana, and Maria José de Córdoba Serrano. 2017. “Musical-Space Synaesthesia: Visualisation of Musical Texture.” Multisensory Research 30 (3–5): 279–285. DOI logoGoogle Scholar
Schaeffer, Pierre. 1966. Traité des objets musicaux [A treatise on musical objects]. Paris: Le Seuil.Google Scholar
Scott, Clive. 2010. “Intermediality and Synesthesia: Literary Translation as Centrifugal Practice.” Art in Translation 2 (2): 153–169. DOI logoGoogle Scholar
Sheets-Johnstone, Maxine. 2011. “Embodied Minds or Mindful Bodies? A Question of Fundamental, Inherently Inter-related Aspects of Animation.” Subjectivity 4 (4): 451–466. DOI logoGoogle Scholar
Silverstein, Michael. 2015. “How Language Communities Intersect: Is ‘Superdiversity’ an Incremental or Transformative Condition?” Language and Communication 441: 7–18. DOI logoGoogle Scholar
Stuart, Susan A. J. 2017. “Feeling Our Way: Enkinesthetic Enquiry and Immanent Intercorporeality.” In Intercorporeality: Emerging Socialities in Interaction, edited by Christian Meyer, Jürgen Streeck, and J. Scott Jordan, 51–72. Oxford: Oxford University Press.Google Scholar
Van Lier, Leo. 2008. “Ecological-Semiotic Perspectives on Educational Linguistics.” In The Handbook of Educational Linguistics, edited by Bernard Spolsky and Francis M. Hult, 596–605. Malden, MA: Blackwell. DOI logoGoogle Scholar
Vatakis, Argiro, and Charles Spence. 2007. “Crossmodal Binding: Evaluating the ‘Unity Assumption’ Using Audiovisual Speech Stimuli.” Perception & Psychophysics 69 (5): 744–756. DOI logoGoogle Scholar
Venuti, Lawrence. 1995. The Translator’s Invisibility: A History of Translation. Abingdon: Routledge.Google Scholar
Ward, Jamie. 2008. The Frog Who Croaked Blue: Synesthesia and the Mixing of the Senses. Hove: Routledge.Google Scholar
Williams, Ralph Vaughan. 1925. “The Lark Ascending: Romance for Violin and Orchestra.” Score. Oxford: Oxford University Press.Google Scholar