Part of
The Pragmatics of Discourse Coherence: Theories and applicationsEdited by Helmut Gruber and Gisela Redeker
[Pragmatics & Beyond New Series 254] 2014
► pp. 179–206
Edited by Helmut Gruber and Gisela Redeker
[Pragmatics & Beyond New Series 254] 2014
► pp. 179–206
Coherence in multimodal arts installations
Multimodal and process-oriented design in digital installation art is pushing for the resolution of physical, spatial and temporal borders as well as coherent structures, which may make visitor perception substantially difficult. Hence, this form of art is well suited for the discussion of the concept of coherence in terms of applicability and constraints. In order to analyse and discuss coherence in multimodal and process-oriented design, I have developed a five-layer framework, based on systemic functional and multimodal theory and capable of analysing coherence relations in the digital installation art that this paper will look at.
References
Arnheim, Rudolf
1969 Visual Thinking. London: University of California Press.
Bateman, John A., and Karl-Heinrich Schmidt
2012 Multimodal Film Analysis: How Films Mean. New York, London: Routledge.
Bateman, John A., and Chiaoi Tseng
2012 “Multimodal Narrative Construction in Christopher Nolan’s Memento a Description of Analytic Method.” Visual Communication 11 (1): 91–119. 
Bobrow, Daniel G., and Donald A. Norman
1975 “Some Principles of Memory Schemata.” In Representation and Understanding: Studies in Cognitive Sciences, ed. by Daniel G. Bobrow, and Allan Collins, 131–149. New York: Academic Press.
Carrell, Patricia L
1982 “Cohesion is not Coherence.” TESOL Quarterly 16 (4): 479–488.
Day, Jon
De Beaugrande, Robert-Alain, and Wolfgang Dressler
1981 Einführung in die Textlinguistik. Tübingen: Niemeyer.
Ehlich, Konrad
2007 Sprache und sprachliches Handeln. Diskurs-Narration-Text-Schrift. Vol. 3. Berlin, New York: Walter de Gruyter.
Halliday, Michael A. K
1978 Language as Social Semiotic. The Interpretation of Language and Meaning. London: Arnold.
2002 “The De-Automatization of Grammar: From Priestly’s An Inspector Calls.” In The Collected Works of M. A. K. Halliday, Vol. 2, 126–148. London: Continuum.
Halliday, Michael A. K., and Ruqaiya Hasan
1976 Cohesion in English. London: Longman.
Hasan, Ruqaiya
1989 Linguistics, Language and Verbal Art. Oxford: Oxford University Press.
Huemer, Birgit
2008a “Representations of Individual and Mass: Modelling Experience through Multiple Modes in Digital Art.” In From Language to Multimodality. New Developments in the Study of Ideational Meaning, ed. by Carys Jones, and Eija Ventola, 255–274. London: Equinox.
2008b “A Semiotic Approach to Visitor Interaction, Participation and Perspective in Digital Arts.” In New Realities: Being Syncretic. Consciousness Reframed: The Planetary Collegium’s IXth International Research Conference Series: Edition Angewandte, ed. by Roy Ascott, Gerald Bast, and Wolfgang Fiel, 144–147. Wien: Springer.
2014 Semiotik der digitalen Medienkunst. Göttingen: V&R unipress.
Jakobson, Roman
1985 “Poetry of Grammar and Grammar of Poetry.” In Jakobson, Roman: Verbal Art, Verbal Sign, Verbal Time, ed. by Krystyna Pomorska, and Stephen Rudy, 37–46. Oxford: Blackwell.
Jewitt, Carey
2005 “Multimodality, ‘‘Reading’’, and ‘‘Writing’’ for the 21st Century.” Discourse: Studies in the Cultural Politics of Education 26 (3): 315–331.
2006 Technology, Literacy and Learning: A Multimodal Approach. Abingdon: Routledge.
Kahn, Douglas
2001 Noise, Water, Meet. A History of Sound in the Arts. Cambridge: MIT Press.
Kamalski, Judith, Ted Sanders, and Leo Lentz
2008 “Coherence Marking, Prior Knowledge, and Comprehension of Informative and Persuasive Texts: Sorting Things Out.” Discourse Processes 45: 323–345.
Kress, Gunther
2003 Literacy in the New Media Age. London: Routledge.
Kress, Gunther, and Theo van Leeuwen
2001 Multimodal Discourse. The Modes and Media of Contemporary Communication. London: Arnold.
2006 Reading Images. The Grammar of Visual Design, 2nd ed. London: Routledge.
Martin, James R
Martin, James R., and David Rose
2003 Working with Discourse. Meaning beyond the Clause. London: Continuum.
Martinec, Radan
1998 “Cohesion in Action.” Semiotica 120 (1/2): 161–180.
2000 “Rhythm in Multimodal Texts.” Leonardo 33 (4): 289–297.
Martinec, Radan, and Andrew Salway
2005 “A System for Image-Text Relations in New (and Old) Media.” Visual Communication 4 (3): 337–371.
Mills, Kathy A
2010 “A Review of the “Digital Turn” in the New Literacy Studies.” Review of Educational Research 80 (2): 246–271.
Morgan, Jerry L., and Manfred Sellner
1980 “Discourse and Linguistic Theory.” In Theoretical Issues in Reading Comprehension: Perspectives from Cognitive Psychology, Linguistics, Artificial Intelligence, and Education, ed. by Rand J. Spiro, Bertram C. Bruce, and William F. Brewer, 165–200. Hillsdale: Lawrence Erlbaum.
Norris, Sigrid
2004 Analyzing Multimodal Interaction: A Methodological Framework. New York: Routledge.
Norris, Sigrid, and Rodney H. Jones
(eds) 2005 Discourse in Action. Introducing Mediated Discourse Analysis. London: Routledge.
O’Toole, Michael
1994 The Language of Displayed Art, new edition. London: Leicester University Press.
Royce, Terry
1999 Visual-Verbal Intersemiotic Complementarity in the Economist Magazine. PhD. Dissertation, University of Reading.
Royce, Terry D., and Wendy L. Bowcher
(eds) 2007 New Directions in the Analysis of Multimodal Discourse. New Jersey: Lawrence Erlbaum.
Sanders, Ted, Jentine Land, and Gerben Mulder
2007 “Linguistic Markers of Coherence Improve Text Comprehension in Functional Contexts.” Information Design Journal 15 (3): 219–235.
Schaefer, Murray R
1986 The Thinking Ear. Toronto: Arcana.
Sperber, Dan, and Deirdre Wilson
1986 Relevance: Communication and Cognition. Oxford: Blackwell.
Spooren, Wilbert, and Ted Sanders
2008 “The Acquisition Order of Coherence Relations: On Cognitive Complexity in Discourse.” Journal of Pragmatics 40 (12): 2003–2026.
Tseng, Chiaoi, and John A. Bateman
2012 “Multimodal Narrative Construction in Christopher Nolan’s Memento a Description of Analytic Method.” Visual Communication 11 (1): 91–119.
Van Leeuwen, Theo
1991 “Conjunctive Structure in Documentary Film and Television.” Journal of Media and Cultural Studies 5 (1): 76–114.
1999 Speech, Music, Sound. London: Palgrave.
2005 Introducing Social Semiotics. London: Routledge.
Data samples in this paper
Bystander
Emotional Traffic (e-traffic)
(2005): Maurice Benayoun and Jean-Baptiste Barrière, music and software by Jean-Baptiste Barrière, software in co-production with V2 Lab, Birgit Lichtenegger, Artem Baguinsky, Marloes de Valk: [Online]: [URL], [visited on 06.08.2012].
Listening Post
Pussy Weevil
Telematic Dreaming
Tempest
The Legible City
The Messenger