Cover not available
Part of
Exploring Discourse Strategies in Social and Cognitive Interaction: Multimodal and cross-linguistic perspectives
Edited by Manuela Romano and Maria Dolores Porto
[Pragmatics & Beyond New Series 262] 2016
► pp. 111–135

The construction of meaning in multimodal discourse

A digital story as a case study

| Technical University of Madrid

This chapter’s main goal is to shed light onto the most characteristic meaning making processes in digital storytelling by thoroughly analyzing a single digital narrative from a functional-cognitive perspective. This mixed approach for a case study allows the analysts to unveil the macro and micro discourse strategies and the cognitive processes developed by both the narrator and the audience to produce and interpret the multiple meanings conveyed by the different modes in a digital story. We believe findings presented here can be of interest for discourse analysts, cognitivists and researchers working in multimodality.

Keywords: digital story, discourse strategies, functional-cognitive approach, multimodality
Published online: 31 March 2016
DOI logo
https://doi.org/10.1075/pbns.262.05mol
References
Alexander, Brian and Alan Levine
2008 “Web 2.0 Storytelling: Emergence of a New Genre”. EDUCAUSE Review 43 (6): 40–48.Google Scholar
Alonso Belmonte, Isabel, Silvia Molina Plaza and M. Dolores Porto Requejo
2013 “Multimodal digital storytelling: Integrating information, emotion and social cognition”. In The Annual Review of Cognitive Linguistics, Special issue “Multimodality and Cognitive Linguistics”, 11 (2): 369–385. Amsterdam/Philadelphia: John Benjamins. DOI logoGoogle Scholar
Baldry, Anthony P. and Paul J. Thibault
2006 Multimodal Transcription and Text Analysis. London and New York: Equinox.Google Scholar
Barthes, Roland
1977 Image, Music, Text. London: Fontana.Google Scholar
Bateman, John A
2008 Multimodality and Genre. A Foundation for the Systematic Analysis of Multimodal Documents. Basingstoke: Palgrave MacMillan.Google Scholar
2014 Text and Image: A Critical Introduction to the Visual/Verbal Divide. London and New York: Routledge.Google Scholar
Bateman, John A. and Janina Wildfeuer
2014 “A Multimodal Discourse Theory of Visual Narrative.” Journal of Pragmatics 74: 180–208. DOI logoGoogle Scholar
Burgess, Jean
2006 “Hearing Ordinary Voices”. Continuum: Journal of Media & Culture 20 (2): 201–214. DOI logoGoogle Scholar
Cohn, Neil
2013 Visual Narrative Structure. Cognitive Science 37 (3): 413–452. DOI logoGoogle Scholar
Dancygier, Barbara
2008 “The text and the story: levels of blending in fictional narratives.” In Mental Spaces in Discourse and Interaction, ed. by Todd Oakley and Anders Hougaard, 51–78. Amsterdam: John Benjamins. DOI logoGoogle Scholar
2012 The Language of Stories: A Cognitive Approach. Cambridge: 
Cambridge University Press.Google Scholar
Fauconnier, Giles and Mark Turner
2002 The Way We Think: Conceptual Blending and the Mind’s Hidden Complexities. New York: Basic Books.Google Scholar
Forceville, Charles and Eduardo Urios-Aparisi
(eds) 2009 Multimodal Metaphor. Berlin: 
Mouton de Gruyter. DOI logoGoogle Scholar
Gleason, Christina
2012 “5 Brands Taking Digital Storytelling to the Next Level”. Ignite Social Media at [URL] (last accessed, January 2015).
Handler Miller, Carolyn
2014 [2004] Digital Storytelling: A Creator’s Guide to Interactive Entertainment. New York and London: Focal Press.Google Scholar
Hühn, Peter, Wolf Schmid and Jörg Schönert
(eds) 2009 Point of View, Perspective, and Focalization: Modeling Mediation in Narrative. Berlin: Mouton de Gruyter. DOI logoGoogle Scholar
Jewitt, Carey
(ed) 2009 The Routledge Handbook of Multimodal Analysis. London: Routledge.Google Scholar
Kövecses, Zoltan
2002 Metaphor: A Practical Introduction. Oxford: Oxford University Press.Google Scholar
Kress, Gunther
2010 Multimodality. A Social Semiotic Approach to Contemporary Communication. London: Routledge.Google Scholar
Kress, Gunther and Theo J. van Leeuwen
2001 Multimodal Discourse: The Modes and Media of Contemporary Communication. London: Arnold.Google Scholar
2006 [1996] Reading Images: The Grammar of Visual Design. London: Routledge.Google Scholar
Lakoff, George and Mark Turner
1989 More than Cool Reason: A Field Guide to Poetic Metaphor. Chicago: University of Chicago Press. DOI logoGoogle Scholar
Lambert, Joe
2013 Digital Storytelling: Capturing Lives, Creating Community. London and New York: Routledge.Google Scholar
Lundby, Knut
(ed) 2008 Digital Storytelling, Mediatized Stories: Self-Representations in New Media. New York: Peter Lang Publishing.Google Scholar
McLellan, Hilary
2007 “Digital Storytelling in Higher Education.” Journal of Computing in Higher Education 19 (1): 65–79. DOI logoGoogle Scholar
McNeill, David
2005 Gesture and Thought. Chicago: Chicago University Press. DOI logoGoogle Scholar
Oakley, Todd and Anders Hougaard
(eds) 2008 Mental Spaces in Discourse and Interaction. Amsterdam: John Benjamins. DOI logoGoogle Scholar
O’Halloran, Kay L
(ed) 2004 Multimodal Discourse Analysis: Systemic Functional Perspectives. London: Bloomsbury Academic.Google Scholar
2008 “Systemic Functional Multimodal Discourse Analysis (SFMDA): Constructing Ideational Meaning Using Language and Visual Imagery.” Visual Communication 7 (4): 443–475. DOI logoGoogle Scholar
O’Halloran, Kay L. and Bradley A. Smith
2011 “Multimodal Text Analysis.” In The Encyclopedia of Applied Linguistics, ed. by Carol A. Chapelle. Oxford, UK: Wiley-Blackwell.Google Scholar
Porto Requejo, M. Dolores
2016. “Music in Multimodal Narratives: The Role of the Soundtrack in Digital Stories”. In Audionarratology: Interfaces of Sound and Narrative ed. by Jarmila Mildorf and Till Kinzel Berlin De Gruyter
Porto, M. Dolores and Manuela Romano
2010 “Conceptual Integration in Natural Oral Narratives”. Actes des Journées D’étude “Narratology and the New Social Dimension of Narrative. [URL] (last accessed January 2015).Google Scholar
Porto Requejo, M. Dolores and Isabel Alonso Belmonte
2014 “From Local to Global: Visual Strategies of Glocalisation in Digital Storytelling.“ Language and Communication 39: 14–23. DOI logoGoogle Scholar
Roland, Craig
2006 Digital Stories in the Classroom. School Art 105 (7): 26.Google Scholar
Rossiter, Marsha and Penny A. Garcia
2010 “Digital Storytelling: A New Player on the Narrative Field.” New Directions for Adult and Continuing Education 126: 37–48. DOI logoGoogle Scholar
Tara La Rose, Janice
2012 “Digital Media Stories through Multimodal Analysis: A Case Study of Erahoneybee’s Song about a Child Welfare Agency.” Journal of Technology in Human Services 30 (3–4): 299–311. DOI logoGoogle Scholar
Tseng, Chiao
2013 Cohesion in film: Tracking film elements. Basingstoke: Palgrave Macmillan. DOI logoGoogle Scholar
Unsworth, Len and Chris Cleirigh
2009 “Multimodality and Reading: The Construction of Meaning through Image-Text Interaction.” In The Routledge Handbook of Multimodal Analysis, ed. by Carey Jewitt, 151–163. London: Routledge.Google Scholar
Velasco, Olga I
2001 “Metaphor, Metonymy, and Image-Schemas: an Analysis of Conceptual Interaction Patterns.” Journal of English Studies 3: 47–63.Google Scholar
Zacks, Jeffrey M. and Barbara Tversky
2001 “Event Structure in Perception and Conception.” Psychological Bulletin 127 (1): 3–21 DOI logoGoogle Scholar
Zwaan, Rolf A
1999 “Five Dimensions of Narrative Comprehension: The Event-Indexing Model.” In Narrative Comprehension, Causality, and Coherence: Essays in Honor of Tom Trabasso, ed. by Susan R. Goldman, Arthur C. Graesser and Paul van den Broek, 93–110. Mahwah, NJ, US: Lawrence Erlbaum.Google Scholar
Zwaan, Rolf A., Mark C. Langston and Arthur C. Graesser
1995 “The Construction of Situation Models in Narrative Comprehension: An Event-Indexing Model.” Psychological Science 6 (5): 292–297. DOI logoGoogle Scholar
Zwaan, Rolf A. and Gabriel A. Radvansky
1998 “Situation Models in Language Comprehension and Memory.” Psychological Bulletin 123 (2): 162–185. DOI logoGoogle Scholar