Perceptual resemblance and the communication of emotion in digital contexts: A case of emoji and reaction GIFs

Ryoko Sasamoto

Abstract

Online communication has created new ways to express emotions, including emoji and reaction GIFs. Emoji are often discussed as signs for meaning-making, adding emotional tone to communication. Reaction GIFs express emotions and attitudes in a playful manner. This study shows that through the lens of cognitive pragmatics, these phenomena are not distinct. Both are cases of non-verbal communication pointing to the communicator’s emotional state. Drawing on relevance-theoretic notions of the showing-meaning continuum and perceptual resemblance, along with relevance-theoretic analyses of metaphor and irony, I argue that emoji and reaction GIFs provide clues to ostension and communicate emotions by virtue of perceptual resemblance between what they represent and the communicator’s emotional state. I will also argue that both emoji and GIFs can involve echoic use of language, enabling the communicator to convey their attitude.

Keywords:
Publication history
Table of contents

The emergence and rapid spread of computer-mediated communication (CMC) in the last few decades has enabled new ways of expressing emotions, including the use of traditional keyboard-based symbols such as emoticons (e.g. :D for a smile), the use of asterisks (e.g. *joy*), acronyms (e.g. LOL), and typographical emphasis, such as the use of uppercase and bold. Others ways of expressing emotions include a visual presentation through digital pictograms or graphic images, such as emoji, stickers or GIFs. In this study, I will focus on emoji and reaction GIFs.

Full-text access is restricted to subscribers. Log in to obtain additional credentials. For subscription information see Subscription & Price. Direct PDF access to this article can be purchased through our e-platform.

Bibliography

Aldredge, Jourdan
2019 “The Michael Jackson Popcorn Gif and the Controversy of ‘Leaving Neverland’.” No Film School. Available at https://​nofilmschool​.com​/Michael​-Jackson​-Popcorn​-GIF. Accessed 15 October 2020.
Ash, James, and Stephen Wilhite
2015 “Sensation, Networks, and the GIF: Toward an Allotropic Account of Affect.” In Networked Affect, ed. by Ken Hillis, Susanna Paasonen, and Michael Petit, 119–133. Cambridge, MA: MIT Press.Google Scholar
Bai, Qiyu, Qi Dan, Zhe Mu, and Maokun Yang
2019 “A Systematic Review of Emoji: Current Research and Future Perspectives.” Frontiers in Psychology 10: 2221. CrossrefGoogle Scholar
Carston, Robyn
1997 “Enrichment and Loosening: Complementary Processes in Deriving the Proposition Expressed?Linguistische Berichte 8, Special Issue on Pragmatics: 103–127. CrossrefGoogle Scholar
1999 “The Semantics/Pragmatics Distinction: A View from Relevance Theory.” In The Semantics/Pragmatics Interface from Different Points of View, ed. by Ken Turner, 85–125. Amsterdam: Elsevier.Google Scholar
2002Thoughts and Utterances. Oxford: Blackwell. CrossrefGoogle Scholar
2004 “Explicature and Semantics.” In Semantics: A Reader, ed. by S. Davis, and B. Gillon, 1–44. Oxford: Oxford University Press.Google Scholar
2008 “Linguistic Communication and the Semantics/Pragmatics Distinctions.” Synthèse 165 (3): 321–345. CrossrefGoogle Scholar
2010 “Metaphor: Ad Hoc Concepts, Literal Meaning and Mental Images.” Proceedings of the Aristotelian Society 110 (3): 295–321. CrossrefGoogle Scholar
Cheng, Lifen
2017 “Do I Mean What I Say and Say What I Mean? A Cross-Cultural Approach to the Use of Emoticons & Emojis in CMC Messages.” FONSECA: Journal of Communication 15: 199–217. CrossrefGoogle Scholar
Cho, Alexander
2015 “Queer Reverb: Tumblr, Affect, Time.” In Networked Affect, ed. by Ken Hillis, Susanna Paasonen, and Michael Petit, 43–58. Cambridge, MA: MIT Press.Google Scholar
Collister, Lauren
Danesi, Marcel
2017The Semiotics of Emoji: The Rise of Visual Language in the Age of the Internet. London: Bloomsbury. CrossrefGoogle Scholar
Eppink, Jason
2014 “A Brief History of the Gif (So Far).” Journal of Visual Culture 13: 298–306. CrossrefGoogle Scholar
Fauconnier, Gilles
1997Mappings in Thought and Language. Cambridge: Cambridge University Press. CrossrefGoogle Scholar
Forceville, Charles
2020Visual and Multimodal Communication: Applying the Relevance Principle. Oxford: Oxford University Press. CrossrefGoogle Scholar
Gibson, Will, Pingping Huang, and Qianyun Yu
2018 “Emoji and Communicative Action: The Semiotics, Sequence and Gestural Actions of ‘Face Covering Hand’.” Discourse, Context & Media 26: 91–99. CrossrefGoogle Scholar
Gülşen, Tüge. T.
2016 “You Tell Me in Emojis.” In Computational and Cognitive Approaches to Narratology, ed. by T. Ogata and T. Akimoto, 354–375. IGI Global. CrossrefGoogle Scholar
Hall, Jeffrey. A., and Natalie Pennington
2013 “Self-Monitoring, Honesty, and Cue Use on Facebook: The Relationship with User Extraversion and Conscientiousness.” Computers in Human Behavior 29: 1556–1564. CrossrefGoogle Scholar
Hjartstrom, Hanna, Daniel E. Sörman, and Jessica K. Ljungberg
2019 “Distraction and Facilitation: The Impact of Emotional Sounds in an Emoji Oddball Task.” PsyCh Journal 8: 180–186. CrossrefGoogle Scholar
Hymes, Dell
1971On Communicative Competence. Philadelphia: University of Pennsylvania Press.Google Scholar
Jaeger, Sara R., and Gastón Ares
2017 “Dominant Meanings of Facial Emoji: Insights from Chinese Consumers and Comparison with Meanings from Internet Resources.” Food Quality and Preference 62: 275–283. CrossrefGoogle Scholar
Jaeger, Sara R., Christina M. Roigard, David Jin, Leticia Vidal, and Gastón Ares
2019 “Valence, Arousal and Sentiment Meanings of 33 Facial Emoji: Insights for the Use of Emoji in Consumer Research.” Food Research International. 119: 895–907. CrossrefGoogle Scholar
Jakobson, Roman
1960 “Linguistics and Poetics.” In Style in Language, ed. by Thomas A. Sebeok, 34–45. Cambridge, MA: MIT Press.Google Scholar
Jibril, Ahmed Tanimu, and Mardziah Haytati Abdullah
2013 “Relevance of Emoticons in Computer-Mediated Communication Contexts: An Overview.” Asian Social Science 9: 201. CrossrefGoogle Scholar
Katz, Yuval, and Limor Shifman
2017 “Making Sense? The Structure and Meanings Of Digital Memetic Nonsense.” Information, Communication & Society 20: 825–842. CrossrefGoogle Scholar
Lakoff, George
1987Women, Fire and Dangerous Things: What Categories Reveal about the Mind. Chicago, IL: University of Chicago Press. CrossrefGoogle Scholar
1994 “What is Metaphor?Advances in Connectionist and Neural Computation Theory 3: 203–258.Google Scholar
López, Rebecca Padilla, and Fabienne Cap
2017 “Did You Ever Read about Frogs Drinking Coffee? Investigating the Compositionality of Multi-Emoji Expressions.” Paper Presented at the 8th Workshop on Computational Approaches to Subjectivity, Sentiment and Social Media Analysis . Crossref
Miltner, Kate M., and Tim Highfield
2017 “Never Gonna GIF You Up: Analyzing the Cultural Significance of the Animated GIF.” Social Media and Society 3.3: 1–11. CrossrefGoogle Scholar
Paasonen, Susanna, Ken Hillis, and Michael Petit
2015 “Introduction: Networks of Transmission: Intensity, Sensation, Value.” In Networked affect, ed. by Ken Hillis, Susanna Paasonen, and Michael Petit, 1–24. Cambridge: The MIT PressGoogle Scholar
Prada, Marília, David L. Rodrigues, Margarida V. Garrido, Diniz Lopes, Bernardo Cavalheiro, and Rui Gaspar
2018 “Motives, Frequency and Attitudes toward Emoji and Emoticon Use.” Telematics and Informatics 35: 1925–1934. CrossrefGoogle Scholar
Riordan, Monica A.
2017 “Emojis as Tools for Emotion Work: Communicating Affect in Text Messages.” Journal of Language and Social Psychology 36: 549–567. CrossrefGoogle Scholar
Sadiq, M., and Shahida
2019 “Learning Pakistani Culture through the Namaz Emoji.” 2nd International Conference on Computing, Mathematics and Engineering Technologies (iCoMET): 1–8, CrossrefGoogle Scholar
Sasamoto, Ryoko and Rebecca Jackson
2016 “Onomatopoeia – Showing-Word or Saying-Word? Relevance Theory, Lexis, and the Communication of Impressions.” Lingua 175: 36–53. CrossrefGoogle Scholar
Sasamoto, Ryoko
2019Onomatopoeia and Relevance Communication of Impressions via Sound. London: Palgrave MacMillan. CrossrefGoogle Scholar
Scott, Kate
2017 “Prosody, Procedures and Pragmatics.” In Semantics and Pragmatics: Drawing a Line, ed. by I. Depraetere, and R. Salkie, 323–341. Berlin: Springer. CrossrefGoogle Scholar
2022Pragmatics Online. London: Routledge. CrossrefGoogle Scholar
Settanni, Michele, and Davide Marengo
2015 “Sharing Feelings Online: Studying Emotional Well-Being Via Automated Text Analysis of Facebook Posts.” Frontiers in Psychology 6: 1045. CrossrefGoogle Scholar
Sha
2016 “The Digital Materiality Of Gifs.” Retrieved from http://​digitalmateriality​.com/
Sperber, Dan, and Deirdre Wilson
1986/1995Relevance: Communication and Cognition, 2nd ed. Oxford: Blackwell.Google Scholar
1987 “Précis of Relevance: Communication and Cognition.” Behavioral and Brain Sciences 10: 697–754. CrossrefGoogle Scholar
1998 “Irony and Relevance: A Reply to Seto, Hamamoto and Yamanashi.” In Relevance Theory. Applications and Implications. ed. by R. Carston, and S. Uchida, 283–293. Amsterdam: John Benjamins. CrossrefGoogle Scholar
2015 “Beyond Speaker’s Meaning.” Croatian Journal of Philosophy 15 (2): 117–149.Google Scholar
2008 “A Deflationary Account of Metaphors.” In The Cambridge Handbook of Metaphor and Thought. ed. by Raymond W. Gibbs, jr., 84–105. Cambridge: Cambridge University Press. CrossrefGoogle Scholar
Sujay, Khandekar, Chae Won Ryu, Joseph Higgs, Jerry O. Talton, Yuanzhe Bian, and Ranjitha Kumar
2019 “Opico: A Study of Emoji-First Communication in a Mobile Social App.” Paper presented at the Companion of the World Wide Web Conference . Crossref
Talmy, Leonard
2000Toward a Cognitive Semantics, Vol. 2. Cambridge, MA: MIT Press.Google Scholar
Tolins, Jackson, and Patrawat Samermit
2016 “Gifs as Embodied Enactments in Text-Mediated Conversation.” Research on Language and Social Interaction 49: 75–91. CrossrefGoogle Scholar
Tossell, Chad C., Philip Kortum, Clayton Shepard, Laura H. Barg-Walkow, Ahmad Rahmati, and Lin Zhong
2012 “A Longitudinal Study of Emoticon Use in Text Messaging from Smartphones.” Computer in Human Behavior 28: 659–663. CrossrefGoogle Scholar
Veszelszki, Agnes
2015 “Emoticons vs. Reaction-Gifs. Non-Verbal Communication on the Internet from the Aspects of Visuality, Verbality and Time.” In Beyond Words. Pictures, Parables, Paradoxes, Visual Learning series, vol. 5, ed. by András Benedek, and Kristóf Nyíri, 131–145. Frankfurt: Peter Lang.Google Scholar
Wharton, Tim
2008 “ ‘Meaning’ and ‘Showing’: Gricean Intentions and Relevance-Theoretic Intentions.” Intercultural Pragmatics 5 (2): 131–152. CrossrefGoogle Scholar
Wharton, Tin
2009Pragmatics and Non-Verbal Communication. Cambridge: Cambridge University Press. CrossrefGoogle Scholar
Wilson, Deirdre, and Dan Sperber
1992 “On Verbal Irony.” Lingua 87: 53–76. CrossrefGoogle Scholar
2002 “Truthfulness and Relevance.” Mind 111 (443): 583–632. CrossrefGoogle Scholar
2004 “Relevance Theory.” In The Handbook of Pragmatics, ed. by Laurence R. Horn, and Gregory Ward, 607–632. Oxford: Blackwell.Google Scholar
2012Relevance and Meaning. Cambridge: Cambridge University Press. CrossrefGoogle Scholar
Wilson, Deirdre, and Robyn Carston
2006 “Metaphor, Relevance and the ‘Emergent Property’ Issue.” Mind & Language 21 (3): 404–433. CrossrefGoogle Scholar
Wilson, Deirdre
2006 “The Pragmatics of Verbal Irony: Echo or Pretence?Lingua 116: 1722–1743. CrossrefGoogle Scholar
2012Metarepresentation in linguistic communication. In Meaning and Relevance, Deidre Wilson, and Dan Sperber, 230–258. Cambridge: Cambridge University Press. CrossrefGoogle Scholar
2021 “Explaining Metonymy.” Paper presented at Institute of Humanities Research Seminar, Northumbria University, 17 March 2021.
Yus, Francisco
2019 “Emoji: A Full Cyberpragmatic Approach.” Paper presented at the 16th China Pragmatics Conference . Nanchang, China.
2021Smartphone Communication: Interactions in the App Ecosystem. London: Routledge. CrossrefGoogle Scholar