Emotions through texts and images
A multimodal analysis of reactions to the Brexit vote on Flickr
We analyzed in multimodal Flickr posts how citizens express emotion in response to the outcome of the EU
Referendum that led to the Brexit vote. We conceived a model that articulates three levels of analysis, in a bid to understand how
meaning operates, namely how inscribed, signalled and/or supported emotion is expressed in narrative and/or conceptual
representations, in image and in text, through logico-semantic relations of expansion, projection and/or decoration. We tested
this model empirically on a corpus of 173 posts. Our results reveal that emotion is very often supported through images and that
narrative representations are particularly prevalent in the text.
Article outline
- 1.Introduction
- 2.Theoretical framework
- 2.1Emotion in affect, judgment and appreciation terms
- 2.2Inscribed, signalled and supported emotion
- 2.3Visual symbols and tropes
- 2.4Four key challenges in text-image analysis
- 3.Data and method
- 3.1Data collection
- 3.2Back and forth between image and text and between models
- 4.Results
- Layers of emotion in image and text
- Set of patterns 1: Visual representations that do not contain any marker of emotion
- Set of patterns 2: Visual representations that inscribe emotion
- Set of patterns 3: Visual representations that support emotion
- 5.Discussion and conclusion
- Acknowledgements
- Note
-
References
References (47)
Ahmed, Sara
2004 ‘
Affective Economies’.
Social Text 79 22 (2): 117–39.
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Alizadeh, Meysam, Ingmar Weber, Claudio Cioffi-Revilla, Santo Fortunato, and Michael Macy
2019 ‘
Psychology and Morality of Political Extremists: Evidence from Twitter Language Analysis of Alt-Right and Antifa’.
Epj Data Science 8 (17).
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Barton, David
2015 ‘
Tagging on Flickr as a Social Pratice’. In
Discourse and Digital Practices: Doing Discourse Analysis in the Digital Age, edited by
Rodney Jones,
Alice Chik, and
Christoph A. Hafner, 48–65. Abingdon, VA: Routledge.
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Bateman, John
2014 Text and Image. A Critical Introduction to the Visual/Verbal Divide. London, New York: Routledge.
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Bateman, John A.
2017 ‘
Triangulating Transmediality: A Multimodal Semiotic Framework Relating Media, Modes and Genres | Elsevier Enhanced Reader’.
Discourse, Context & Media, 160–74.
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Bednarek, Monika
2008 Emotion Talk Across Corpora. Basingstoke, UK: Palgrave MacMillan.
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Bouko, Catherine, July De Wilde, Sofie Decock, Valentina Manchia, Oprhée De Clercq, and David Garcia
2018 ‘
Reactions to Brexit in Images: A Multimodal Content Analysis of Shared Visual Content on Flickr’.
Visual Communication.
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Bourlai, Elli, and Susan C. Herring
2014 ‘
Multimodal Communication on Tumblr: I Have so Many Feels!’ In
Proceedings of the 2014 ACM Conference on Web Science, 171–175. Bloomington, IN: ACM.
[URL]
Burgess, Jean
2011 ‘
Image Sharing in the #qldfloods’.
Mapping Online Publics 6 March 2011
[URL]
Burke, Kenneth
1969 A Grammar of Motives. Berkeley: University of California Press.
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Caple, Helen
2018 ‘
“Lucy Says Today She Is a Labordoodle”: How the Dogs-of-Instagram Reveal Voter Preferences’.
Social Semiotics 0 (0): 1–21.
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Celli, Fabio, Evgeny A. Stepanov, Massimo Poesio, and Giuseppe Riccardi
2016 ‘
Predicting Brexit: Classifying Agreement Is Better than Sentiment and Pollsters’. presented at the Proceedings PEOPLE Workshop, Osaka.
[URL]
Chouliaraki, Lilie
2017 ‘
Towards an Analytics of Mediation’. In
The Routledge Handbook of Multimodal Analysis, edited by
Carey Jewitt, 2nd ed., 253–73. Abingdon: Routledge.
[URL]
Del Vicario, Michela, Fabiana Zollo, Guido Caldarelli, Antonio Scalla, and Walter Quattrociocchi
2017 ‘
Mapping Social Dynamics on Facebook: The Brexit Debate’.
Social Networks, no. 50: 6–16.
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Garcia, David, and Bernard Rimé
2019 ‘
Collective Emotions and Social Resilience in the Digital Traces After a Terrorist Attack’.
Psychological Science 30 (4): 617–28.
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Gaspar, Rui, Claudia Pedro, Panos Panagiotopoulos, and Beate Seibt
2016 ‘
Beyond Positive or Negative: Qualitative Sentiment Analysis of Social Media Reactions to Unexpected Stressful Events’.
Computers in Human Behavior 561 (
March): 179–91.
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Giaxoglou, Korina
2019 ‘
Visual Small Stories of #jesuisaylan: Sharing Cosmopolitan Emotions on Instagram’.
Cahiers de l’ILSL 591: 59–75.
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Halliday, Michael
1985 An Introduction to Functional Grammar. London: Edward Arnold.
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Halliday, Michael, and Christian Matthiessen
2004 An Introduction to Functional Grammar. 3rd edition. London: Routledge.
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Harris, Ranida B., and David Paradice
2007 ‘
An Investigation of the Computer-Mediated Communication of Emotions’.
Journal of Applied Sciences Research 3 (12): 2081–90.
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Kharroub, Tamara, and Ozen Bas
2016 ‘
Social Media and Protests: An Examination of Twitter Images of the 2011 Egyptian Revolution’.
New Media & Society 18 (9): 1973–92.
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Kouper, Inna
2010 ‘
The Pragmatics of Peer Advice in a LiveJournal Community’.
Language@Internet.
[URL]
Kress, Gunther, and Theo van Leeuwen
2006 Reading Images: The Grammar of Visual Design (2nd Edition). New York: Routledge.
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Lakoff, George
1993 ‘
Metaphor and Thought’. In
The Contemporary Theory of Metaphor, edited by
Andrew Ortony, 202–51. Cambridge: Cambridge University Press.
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Lansdall-Welfare, Tom, Fabom Dzogang, and Nello Cristianini
2016 ‘
Change-Point Analysis of the Public Mood in UK Twitter during the Brexit Referendum’. In
2016 IEEE 16th International Conference on Data Mining Workshops (ICDMW). Barcelona, Spain: IEEE.
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Liu, Sophia, Jeannette Sutton, Amanda Hughes, and Sarah Vieweg
2008 ‘
In Search of the Bigger Picture: The Emergent Role of On-Line Photo Sharing in Times of Disaster’. In
Proceedings of the 5th International ISCRAM Conference.
[URL]
Llewellyn, Clare, and Laura Cram
2016 ‘
The Results Are in and the UK Will #Brexit: What Did Social Media Tell Us about the UK’s EU Referendum?’ In
EU Referendum Analysis 2016: Media, Voters and the Campaign. Early Reflections from Leading UK Academics, edited by
Daniel Jackson,
Einar Thorsen, and
Dominic Wring, 90–91. Poole: The Centre for the Study of Journalism, Culture and Community Bournemouth University.
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Löbner, Sebastian
2002 Understanding Semantics. London: Arnold.
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Lyons, John
1995 Linguistic Semantics. Cambridge: Cambridge University Press.
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Machin, David
2007 Introduction to Multimodal Analysis. London: Bloomsbury Academic.
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Mann, William C., and Sandra A. Thompson
1988 ‘
Rethorical Structure Theory: Toward a Functional Theory of Text Organization’.
Text – Interdisciplinary Journal for the Study of Discourse 8 (3): 243–81.
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Manovich, Lev
2016 Instagram and Contemporary Image.
[URL]
Martin, J. R., and P. R. R. White
2005 The Language of Evaluation. Appraisal in English. London: Palgrave Macmillan.
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Micheli, Raphaël
2014 Les Émotions Dans Les Discours. Modèle d’analyse, Perspectives Empiriques. Bruxelles: De Boeck.
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Papacharissi, Zizi
(
2015)
Affective Publics: Sentiment, Technology, and Politics. Oxford: Oxford University Press.
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Papacharissi, Zizi, and Maria de Fatima Oliveira
2012 ‘
Affective News and Networked Publics: The Rhythms of News Storytelling on #Egypt’.
Journal of Communication 62 (2): 266–82.
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Plantin, Christian
2011 Les bonnes raisons des émotions: principes et méthode pour l’étude du discours émotionné. Berne: Peter Lang.
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Royce, Terry D.
2007 ‘
Intersemiotic Complementarity: A Framework for Multimodal Discourse Analysis’. In
New Directions in the Analysis of Multimodal Discourse, edited by
Terry D. Royce and
Wendy Bowcher, 63–109. New York: Lawrence Erlbaum & Assoc.
[URL]
Seko, Yukari
2013 ‘
Picturesque Wounds: A Multimodal Analysis of Self-Injury Photographs on Flickr’.
Forum Qualitative Sozialforschung / Forum: Qualitative Social Research 14 (2).
[URL]
Seltzer, E. K., N. S. Jean, E. Kramer-Golinkoff, D. A. Asch, and R. M. Merchant
2015 ‘
The Content of Social Media’s Shared Images about Ebola: A Retrospective Study’.
Public Health 129 (9): 1273–77.
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Shaver, Phillip, Judith Schwartz, Donald Kirson, and Cary O’Connor
1987 ‘
Emotion Knowledge: Further Exploration of a Prototype Approach’.
Journal of Personality and Social Psychology 52 (6): 1061–86.
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Skinner, Ellen A., Kathleen Edge, Jeffrey Altman, and Hayley Sherwood
2003 ‘
Searching for the Structure of Coping: A Review and Critique of Category Systems for Classifying Ways of Coping’.
Psychological Bulletin 129 (2): 216–269.
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Smith, Kit
2017 ‘
44 Incredible and Interesting Twitter Statistics’.
Brandwatch (blog)
17 December 2017
[URL]
Van Dijck, José
2011 ‘
Flickr and the Culture of Connectivity: Sharing Views, Experiences, Memories’.
Memory Studies 4 (4): 401–15.
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Varol, Onur, Emilio Ferrara, Clayton Davis, Filippo Menczer, and Alessandro Flammini
2017 ‘
Online Human-Bot Interactions: Detection, Estimation, and Characterization’. In
Eleventh International AAAI Conference on Web and Social Media.
[URL]
Vis, Farida, S. Faulkner, K. Parry, Yana Manyukhina, and L. Evans
2014 ‘
Twitpic-Ing the Riots. Analysing Images Shared on Twitter during the 2011 UK Riots’. In
Twitter and Society, edited by
Katrin Weller,
Axel Bruns,
Jean Burgess,
Merja Mahrt, and
Cornelius Puschmann, 385–98. New York: Peter Lang.
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Cited by (1)
Cited by 1 other publications
Trushchelev, Pavel N., Elena V. Petrenko & Larisa A. Piotrovskaya
2023.
Conceptualizing emotions through discourse: a pragmatic view on the reader's interest.
Slovo.ru: Baltic accent 14:2
► pp. 93 ff.
![DOI logo](//benjamins.com/logos/doi-logo.svg)
This list is based on CrossRef data as of 5 july 2024. Please note that it may not be complete. Sources presented here have been supplied by the respective publishers.
Any errors therein should be reported to them.