Cover not available
Part of
Media as Procedures of Communication
Edited by Martin Luginbühl and Jan Georg Schneider
[Pragmatics & Beyond New Series 348] 2024
► pp. 16–39

Chapter 2
Mediality vs. materiality

A multimodal perspective on the notion of media as procedures

| University of Groningen
This contribution adds a multimodal perspective to the notions of medium and mediality in the context of media linguistics and critically reflects upon their conceptualisations as ‘technical infrastructures’ and/or ‘socially constituted semiotic procedures’. By critically assessing the different understandings of medium in media linguistics and multimodality research, it will point out advantages and disadvantages of each concept. In a second step, the definitions will be applied to practical examples of current media artifacts, i.e., a graphic novel and a digital learning environment. The application will demonstrate the need for a clear distinction between various levels of description for both semiotic modes as well as media.
Keywords: medium, mediality, multimodality, semiotic mode, materiality, canvas
Article outline
  • 1.Introduction
  • 2.Two understandings of ‘medium’
    • 2.1Media as procedures of sign processing
    • 2.2Media as sites for the deployment of semiotic modes
  • 3.Media and modes at work
    • 3.1Sign processing in comics
    • 3.2Sign processing in digital learning environments
  • 4.The mutual influence of medium and modes
  • Notes
  • References
This content is being prepared for publication; it may be subject to changes.
References (39)
References
Baldrauf-Quilliatre, Heike, and Isabel Colón de Carvajal. 2019. “Encouragement in Videogame Interactions.” Social Interaction. Video-Based Studies of Human Sociality 2(2). DOI logoGoogle Scholar
Bateman, John A. 2013. “Multimodality and Film.” In The Encyclopedia of Applied Linguistics, edited by Carol A. Chapelle. London: Blackwell Publishing. DOI logoGoogle Scholar
2016. “Methodological and Theoretical Issues for the Empirical Investigation of Multimodality.” In Handbuch Sprache im multimodalen Kontext, edited by Nina-Maria Klug, and Hartmut Stöckl, 36–74. Berlin: de Gruyter. DOI logoGoogle Scholar
2017. “Triangulating Transmediality: A Multimodal Semiotic Framework Relating Media, Modes and Genres.” Discourse, Context & Media 20: 160–174. DOI logoGoogle Scholar
2020. “When Holism Comes Back to Bite You: Comments on “Medial Shaping from the Outset” by Martin Luginbühl, and Jan Georg Schneider.” Journal für Medienlinguistik/Journal for Media Linguistics – Open Peer Review. [URL]
2021. “What are Digital Media?” Discourse, Context & Media 41: 100502. DOI logoGoogle Scholar
Bateman, John A., and Janina Wildfeuer. 2014a. “A Multimodal Discourse Theory of Visual Narrative.” Journal of Pragmatics 74: 180–218. DOI logoGoogle Scholar
. 2014b. “Defining Units of Analysis for the Systematic Analysis of Comics: A Discourse-Based Approach.” Studies in Comics 5(2): 371–401. DOI logoGoogle Scholar
Bateman, John A., Janina Wildfeuer, and Tuomo Hiippala. 2017. Multimodality. Foundations, Research, Analysis. A Problem-Oriented Discussion. Berlin: de Gruyter. DOI logoGoogle Scholar
Bender, Michael, Ruth M. Mell, and Janina Wildfeuer. 2022. “Zur Spezifik digitaler Medien als Diskursraum: Materialität, Daten, Affordanzen.” In Diskurse — digital. Theorien, Methoden, Fallstudien, edited by Eva Gredel, 27–45. Berlin, Boston: de Gruyter. DOI logoGoogle Scholar
Berlanga-Fernandez, Inmaculada, and Everardo Reyes. 2022. “The Digital Approach to Semiotics: A Systematic Review.” Text & Talk, ahead of print. DOI logoGoogle Scholar
Brock, Alexander, and Peter Schildhauer (eds). 2017. Communication Forms and Communicative Practices. New Perspectives on Communication Forms, Affordances and What Users Make of Them. Bern, Berlin: Lang.Google Scholar
Cohn, Neil. 2013. The Visual Language of Comics. Introduction to the Structure and Cognition of Sequential Images. New York: Bloomsbury.Google Scholar
Cohn, Neil, and Stephen Maher. 2015. “The Notion of the Motion. The Neurocognition of Motion Lines in Visual Narratives.” Brain Research 160: 73–84. DOI logoGoogle Scholar
Elleström, Lars. 2010. Media Borders. Multimodality and Intermediality. London: Palgrave Macmillan. DOI logoGoogle Scholar
Frobenius, Maximiliane (2014). “Audience Design in Monologues: How Vloggers Involve Their Viewers.” Journal of Pragmatics 72: 59–72. DOI logoGoogle Scholar
Holly, Werner. 2016. “Nachrichtenfilm als multimodale audiovisuelle Texte.” In Handbuch Sprache im multimodalen Kontext, edited by Nina-Maria Klug, and Hartmut Stöckl, 392–409. Berlin: de Gruyter. DOI logoGoogle Scholar
Horton, Donald, and Wohl, Richard R. (1956). “Mass Communication and Para-Social Interaction: Observations on Intimacy at a Distance.” Psychiatry 19: 215–229. DOI logoGoogle Scholar
Klug, Nina-Maria, and Hartmut Stöckl. 2016. Handbuch Sprache im multimodalen Kontext. Berlin: de Gruyter. DOI logoGoogle Scholar
Kress, Gunther, and Theo van Leeuwen. 2001. Multimodal Discourse. The Modes and Media of Contemporary Communication. London: Arnold.Google Scholar
Luginbühl, Martin, and Jan Georg Schneider. 2020. “Medial Shaping from the Outset. On the Mediality of the Second Presidential Debate, 2016.” Journal für Medienlinguistik 3(1): 57–93. DOI logoGoogle Scholar
Mawil (2014). Kinderland. Berlin: Reprodukt.Google Scholar
McCloud, Scott. 1994. Understanding Comics: The Invisible Art. New York: HarperPerennial.Google Scholar
Nake, Frieder. 1994. “Human-Computer Interaction Viewed as Pseudo-Communication.” Knowledge-Based Systems 14: 441–447. DOI logoGoogle Scholar
O’Halloran, Kay L. 2004. “Visual Semiosis in Film.” In Multimodal Discourse Analysis: Systemic Functional Perspectives, edited by Kay L. O’Halloran, 109–130. London: Continuum.Google Scholar
Packard, Stephan, Andreas Rauscher, Veronique Sina, Jan-Noel Thon, Lukas Wilde, and Janina Wildfeuer. 2019. Comicanalyse. Eine Einführung. Stuttgart: J.B. Metzler. DOI logoGoogle Scholar
Ryan, Marie-Laure. 2004. Narrative across Media: The Languages of Storytelling. Nebraska: University of Nebraska Press.Google Scholar
Schmitz, Ulrich. 2018. “Media Linguistic Landscapes. Alle Linguistik sollte Medienlinguistik sein.” Journal Für Medienlinguistik / Journal for Media Linguistics 1(1): 1–34. DOI logoGoogle Scholar
. 2022. “Kann und soll Linguistik angesichts multimodaler Kommunikation eine eigene Disziplin sein?” Osnabrücker Beiträge zur Sprachtheorie 100: 11–28. DOI logoGoogle Scholar
Schneider, Jan Georg. 2008. Spielräume der Medialität. Linguistische Gegenstandskonstitution aus medientheoretischer und pragmatischer Perspektive. Berlin, New York: de Gruyter.Google Scholar
. 2017. “Medien als Verfahren der Zeichenprozessierung. Grundsätzliche Überlegungen zum Medienbegriff und ihre Relevanz für die Gesprächsforschung.” Gesprächsforschung — Online-Zeitschrift zur verbalen Interaktion 18: 34–55. [URL]Google Scholar
Unsworth, Len. 2007. “Image/Text Relations and Intersemiosis: Towards Multimodal Text Description for Multiliteracies Education.” In Proceedings of the 33rd International Systemic Functional Congress (33rd ISFC), edited by Leila Barbara, and Tony B. Sardinha, 1165–1205. [URL]
Wildfeuer, Janina. 2012. “Intersemiosis in Film: Towards a New Organisation of Semiotic Resources in Multimodal Filmic Text.” Multimodal Communication 1(3): 233–304. DOI logoGoogle Scholar
. 2018. “It’s All about Logics?! Analyzing the Rhetorical Structure of Multimodal Filmic Text.” Semiotica 220: 95–121. DOI logoGoogle Scholar
. 2019. “The Inferential Semantics of Comics: Panels and their Meanings.” Poetics Today 40(2): 215–234. DOI logoGoogle Scholar
. 2021. “Discourse Semantics and Textual Logic: Methodological Considerations for Multimodal Analysis.” In OBST — Osnabrücker Beiträge zur Sprachtheorie 99. Special Issue on Linguistic and Multimodality, edited by Arne Krause, and Ulrich Schmitz, 87–114.Google Scholar
Wildfeuer, Janina, and John A. Bateman. 2018. “Theoretische und methodologische Perspektiven des Multimodalitätskonzepts aus linguistischer Sicht.” IMAGE — Zeitschrift Für Interdisziplinäre Bildwissenschaft 28: 5–46.Google Scholar
Wildfeuer, Janina, Jana Pflaeging, John A. Bateman, Ognyan Seizov, and Chiao-I Tseng. 2019. “Multimodality: Disciplinary Thoughts and the Challenge of Diversity. Introduction.” In Multimodality: Disciplinary Thoughts and the Challenge of Diversity, edited by Janina Wildfeuer, Jana Pflaeging, John A. Bateman, Ognyan Seizov, and Chiao-I Tseng, 3–38. Berlin: de Gruyter. DOI logoGoogle Scholar
Winkler, Hartmut. 2008. Basiswissen Medien. Frankfurt a.M.: FischerGoogle Scholar