Cover not available
Part of
Figurative Language – Intersubjectivity and Usage
Edited by Augusto Soares da Silva
[Figurative Thought and Language 11] 2021
► pp. 109–136

The multimodal negotiation of irony and humor in interaction

On the role of eye gaze in joint pretense

| University of Leuven

Interactionally grounded accounts of humor and irony have focused on the construction of complex layered gestalts. In most cases, these accounts provide a model for the pretense that speakers are engaged in when jointly construing ironic or humorous utterances, as well as for the affective power of such utterances. Much less studied, however, is the question how speakers interactionally monitor such sequences of joint pretense. To investigate this more systematically, I zoom in on the role of eye gaze as a mechanism for reaction monitoring by speakers and hearers. Using humorous sequences taken from a multimodal video corpus of three-party interactions, in which the gaze behavior of all participants was recorded using mobile eye-tracking devices, I describe specific gaze patterns.

Keywords: irony, humor, pretense, eye gaze, eye-tracking, multimodality
Article outline
  • 1.Introduction
  • 2.Irony and humor in interaction
  • 3.Eye gaze in interaction
  • 4.Eye gaze and interactional humor
  • 5.Research questions and data set
  • 6.A micro-analysis of selected sequences
  • 7.Concluding remarks
  • Acknowledgements
  • Notes
  • References
Published online: 19 May 2021
DOI logo
https://doi.org/10.1075/ftl.11.04bro
References
Argyle, M., & Cook, M.
(1976) Gaze and mutual gaze. London: Cambridge University Press.Google Scholar
Athanasiadou, A., & Colston, H. L.
(Eds.) (2017) Irony in language use and communication. Amsterdam: John Benjamins. DOI logoGoogle Scholar
Attardo, S.
(1994) Linguistic theories of humor. Berlin/New York: Mouton de Gruyter.Google Scholar
(2000) Irony as relevant inappropriateness. Journal of Pragmatics, 32 (6), 793–826. DOI logoGoogle Scholar
(2001a) Humorous texts: A semantic and pragmatic analysis. Berlin/New York: Mouton de Gruyter. DOI logoGoogle Scholar
(2001b) Humor and irony in interaction: From mode adoption to failure of detection. In L. Anolli, R. Ciceri, & G. Riva (Eds.), Say not to say: New perspectives on miscommunication (pp. 165–185). Amsterdam: IOS Press.Google Scholar
(Ed.) (2017) The Routledge handbook of language and humor. London: Routledge. DOI logoGoogle Scholar
Auer, P.
(2018) Gaze, addressee selection and turn-taking in three-party interaction. In G. Brône, & B. Oben (Eds.), Eye-tracking in interaction. Studies on the role of eye gaze in dialogue. (pp. 197–232). Amsterdam/Philadelphia: John Benjamins. DOI logoGoogle Scholar
Barnden, J.
(2017) Irony, pretence and fictively-elaborating hyperbole. In A. Athanasiadou, & H. L. Colston (Eds.), Irony in language use and communication (pp. 145–178). Amsterdam: John Benjamins. DOI logoGoogle Scholar
Bavelas, J., Coates, L., & Johnson, T.
(2002) Listener responses as a collaborative process: The role of gaze. Journal of Communication, 52 (3), 566–580. DOI logoGoogle Scholar
Beukeleers, I., Brône, G., & Vermeerbergen, M.
(2020) Unaddressed participants’ gaze behavior in Flemish Sign Language interactions: Planning gaze shifts after recognizing an upcoming (possible) turn completion. Journal of Pragmatics, 162 , 62–83. DOI logoGoogle Scholar
Brône, G.
(2008) Hyper- and misunderstanding in interactional humor. Journal of Pragmatics, 40 (12), 2027–2061. DOI logoGoogle Scholar
(2012) Humour and irony in cognitive pragmatics. In H. J. Schmid (Ed.), Handbook of cognitive pragmatics (pp. 463–504). Berlin: Mouton de Gruyter. DOI logoGoogle Scholar
(2017) Cognitive Linguistics and humor research. In S. Attardo (Ed.), The Routledge handbook of language and humor (pp. 250–266). London: Routledge. DOI logoGoogle Scholar
Brône, G., & Oben, B.
(2015) InSight interaction: A multimodal and multifocal dialogue corpus. Language Resources and Evaluation, 49 (1), 195–214. DOI logoGoogle Scholar
Brône, G., Oben, B., Vranjes, J., & Feyaerts, K.
(2017) Eye gaze and viewpoint in multimodal interaction management. Cognitive Linguistics, 28 (3), 449–483. DOI logoGoogle Scholar
Brône, G., & Oben, B.
(2018) Introduction: Gaze, interaction and eye-tracking: A multidisciplinary endeavor. In G. Brône, & B. Oben (Eds.), Eye-tracking in interaction. Studies on the role of eye gaze in dialogue (pp. 1–20). Amsterdam/Philadelphia: John Benjamins. DOI logoGoogle Scholar
Brugman, H., & Russel, A.
(2004) Annotating Multimedia/Multi-modal resources with ELAN. In: Proceedings of the fourth International Conference on Language Resources and Evaluation (LREC).Google Scholar
Chafe, W.
(1994) Discourse, consciousness, and time: The flow and displacement of conscious experience in speaking and writing. Chicago: University of Chicago Press.Google Scholar
Cienki, A.
(2016) Cognitive linguistics, gesture studies, and multimodal communication. Cognitive Linguistics, 27 (4), 603–618. DOI logoGoogle Scholar
(2017) Analysing metaphor in gesture: A set of metaphor identification guidelines for gesture (MIG-G). In E. Semino, & Z. Demjén (Eds.), The Routledge handbook of metaphor and language (pp. 131–147). London: Routledge.Google Scholar
Cienki, A., & Müller, C.
(Eds.) (2008) Metaphor and gesture. Amsterdam/Philadelphia: John Benjamins. DOI logoGoogle Scholar
Clark, H. H.
(1996) Using language. Cambridge: Cambridge University Press. DOI logoGoogle Scholar
(2016) Depicting as a method of communication. Psychological Review, 123 (3), 324–347. DOI logoGoogle Scholar
Clark, H. H., & Gerrig, R.
(1984) On the pretense theory of irony. Journal of Experimental Psychology: General, 113 , 121–126. DOI logoGoogle Scholar
Colston, H. L.
(2017) Irony performance and perception: What underlies verbal, situational and other ironies? In A. Athanasiadou, & H. L. Colston (Eds.), Irony in language use and communication (pp. 19–42). Amsterdam: John Benjamins. DOI logoGoogle Scholar
Coulson, S.
(2005) Sarcasm and the Space Structuring Model. In S. Coulson, & B. Lewandowska-Tomaszczyk (Eds.), The literal and the nonliteral in language and thought (pp. 129–144). Berlin: Peter Lang.Google Scholar
Coulson, S., Urbach, T. P., & Kutas, M.
(2006) Looking back: Joke comprehension and the space structuring model. Humor. International Journal of Humor Research, 19 (3), 229–250. DOI logoGoogle Scholar
Dancygier, B., & Sweetser, E.
(Eds.) (2012) Viewpoint in language. A multimodal perspective. Cambridge: Cambridge University Press. DOI logoGoogle Scholar
Deppermann, A.
(2013) Multimodal interaction from a conversation analytic perspective. Journal of Pragmatics, 46 , 1–7. DOI logoGoogle Scholar
Deppermann, A., & Günthner, S.
(Eds.) (2015) Temporality in interaction. Amsterdam: John Benjamins. DOI logoGoogle Scholar
Deppermann, A., & Streeck, J.
(2018) Time in embodied interaction: Synchronicity and sequentiality of multimodal resources. Amsterdam: John Benjamins. DOI logoGoogle Scholar
Devylder, S.
(2019) Review of Barbara Dancygier, ed., The Cambridge Handbook of Cognitive Linguistics. English Text Construction, 12 (1), 144–154. DOI logoGoogle Scholar
Duncan, S.
(1975) Interaction units during speaking turns in dyadic, face-to-face conversations. In A. Kendon, R. M. Harris, & M. R. Key (Eds.), Organization of behavior in face-to-face interaction (pp. 199–212). The Hague: Mouton. DOI logoGoogle Scholar
Fauconnier, G.
(1994) Mental spaces: Aspects of meaning construction in natural language. Cambridge (MA): MIT Press. DOI logoGoogle Scholar
(1997) Mappings in thought and language. Cambridge: Cambridge University Press. DOI logoGoogle Scholar
Fauconnier, G., & Turner, M.
(2002) The way we think. Conceptual blending and the mind’s hidden complexities. New York: Basic Books.Google Scholar
Ferstl, E. C., Israel, L., & Putzar, L.
(2016) Humor facilitates text comprehension: Evidence from eye movements. Discourse Processes, 54 (4), 259–284. DOI logoGoogle Scholar
Feyaerts, K., Brône, G., & Oben, B.
(2017) Multimodality in Interaction. In B. Dancygier (Ed.), The Cambridge handbook of Cognitive Linguistics (pp. 135–156). Cambridge: Cambridge University Press. DOI logoGoogle Scholar
Filik, R., Brightman, E., Gathercole, C., & Leuthold, H.
(2017) The emotional impact of verbal irony: Eye-tracking evidence for a two-stage process. Journal of Memory and Language, 93 , 193–202. DOI logoGoogle Scholar
Gibbs, R. W. Jr.
(1994) The poetics of mind: Figurative thought, language, and understanding. Cambridge/New York: Cambridge University Press.Google Scholar
(2000) Irony in talk among friends. Metaphor and Symbol, 15 , 5–27. DOI logoGoogle Scholar
Gibbs, R. W. Jr., & Colston, H.
(Eds.) (2007) Irony in language and thought. A cognitive science reader. London: Routledge. DOI logoGoogle Scholar
Gibbs, R. W. Jr., Bryant, G., & Colston, H. L.
(2014) Where is the humor in verbal irony? Humor. International Journal of Humor Research, 27 (4), 575–595. DOI logoGoogle Scholar
Giora, R.
(2003) On our mind: Salience, context and figurative language. New York: Oxford University Press. DOI logoGoogle Scholar
Giora, R., Fein, O., Laadan, D., Wolfson, J., Zeituny, M., Kidron, R., Kaufman, R., & Shaham, R.
(2007) Expecting irony: Context vs. salience-based effects. Metaphor and Symbol, 22 , 119–146. DOI logoGoogle Scholar
Giora, R., & Attardo, S.
(2014) Irony. In S. Attardo (Ed.), Encyclopedia of humor studies (pp. 397–401). Thousand Oaks: SAGE.Google Scholar
Giora, R., Fein, O., & Schwartz, T.
(1998) Irony: graded salience and indirect negation. Metaphor and Symbol, 13 , 83–101. DOI logoGoogle Scholar
Gironzetti, E., Pickering, L., Huang, M., Zhang, Y., Menjo, S., & Attardo, S.
(2016) Smiling synchronicity and gaze patterns in dyadic humorous conversations. Humor: International Journal of Humor Research, 29 (2), 301–324. DOI logoGoogle Scholar
Goodwin, C.
(1980) Restarts, pauses, and the achievement of a state of mutual gaze. Sociological Inquiry, 50 (3–4), 272–302. DOI logoGoogle Scholar
(1981) Conversational organization. Interaction between speakers and hearers. New York/London: Academic Press.Google Scholar
(2007) Interactive footing. In E. Holt, & R. Clift (Eds.), Reporting talk: Reported speech in interaction (pp. 16–46). Cambridge: Cambridge University Press.Google Scholar
(2017) Cooperative action. Cambridge: Cambridge University Press. DOI logoGoogle Scholar
Goodwin, M. H., & Goodwin, C.
(1986) Gesture and coparticipation in the activity of searching for a Word. Semiotica, 62 , 51–75.Google Scholar
Gullberg, M., & Kita, S.
(2009) Attention to speech-accompanying gestures: Eye movements and information uptake. Journal of Nonverbal Behaviour, 33 , 251–277. DOI logoGoogle Scholar
Haddington, P.
(2006) The organization of gaze and assessments as resources for stance taking. Text & Talk, 26 (3), 281–328. DOI logoGoogle Scholar
Hirvenkari, L., Ruusuvuori, J., Saarinen, V. -M., Kivioja, M., Peräkylä, A., et al.
(2013) Influence of turn-taking in a two-person conversation on the gaze of a viewer. PLoS ONE, 8 (8), 2–6. DOI logoGoogle Scholar
Holler, J., & Kendrick, K.
(2015) Unaddressed participants’ gaze in multi-person interaction: Optimizing recipiency. Frontiers in Psychology, 6 , 1–14. DOI logoGoogle Scholar
Kendon, A.
(1967) Some functions of gaze-direction in social interaction. Acta Psychologica, 26 , 22–63. DOI logoGoogle Scholar
Kidwell, M.
(2005) Gaze as social control: How very young children differentiate “The Look” from a “Mere Look” by their adult caregivers. Research on Language and Social Interaction, 38 (4), 417–449. DOI logoGoogle Scholar
(2009) Gaze shift as an interactional resource for very young children. Discourse Processes, 46 , 145–160. DOI logoGoogle Scholar
Kihara, Y.
(2005) The mental space structure of verbal irony. Cognitive Linguistics, 16 (3), 513–530. DOI logoGoogle Scholar
Kreuz, R. J., & Glucksberg, S.
(1989) How to be sarcastic: The reminder theory of verbal irony. Journal of Experimental Psychology: General, 118 , 347–386.Google Scholar
Kumon-Nakamuro, S., Glucksberg, S., & Brown, M.
(1995) How about another piece of pie: the allusional pretense theory of discourse irony. Journal of Experimental Psychology: General, 124 (1), 3–21. DOI logoGoogle Scholar
Lausberg, H., & Sloetjes, H.
(2009) Coding gestural behavior with the NEUROGES-ELAN system. Behavior Research Methods, Instruments, & Computers, 41 , 841–849. DOI logoGoogle Scholar
Linell, P.
(2009) Rethinking language, mind, and world dialogically. London: Information Age Publishing.Google Scholar
López, B. G., & Vaid, J.
(2017) Psycholinguistic approaches to humor. In S. Attardo (Ed.), The Routledge handbook of language and humor (pp. 267–281). London: Routledge. DOI logoGoogle Scholar
McClave, E. Z.
(2000) Linguistic functions of head movements in the context of speech. Journal of Pragmatics, 32 , 855–878. DOI logoGoogle Scholar
Mondada, L.
(2019) Contemporary issues in conversation analysis: Embodiment and materiality, multimodality and multisensoriality in social interaction. Journal of Pragmatics, 145 , 47–62. DOI logoGoogle Scholar
Mortensen, K.
(2012) Conversation analysis and multimodality. In J. Wagner, & J. K. Mortensen (Eds.), Conversation analysis and applied linguistics. The encyclopedia of applied linguistics (pp. 1061–1068). Oxford: Wiley-Blackwell.. DOI logoGoogle Scholar
Mulder, J., & Thompson, S. A.
(2008) The grammaticalization of but as a final particle in English conversation. In R. Laury (Ed.), Crosslinguistic studies of clause combining: The multifunctionality of conjunctions (pp. 179–204). Amsterdam/Philadelphia: John Benjamins. DOI logoGoogle Scholar
Parrill, F.
(2012) Interactions between discourse status and viewpoint in co-speech gesture. In B. Dancygier, & E. Sweetser (Eds.), Viewpoint in language: A multimodal perspective (pp. 97–112). Cambridge: Cambridge University Press. DOI logoGoogle Scholar
Pickering, M., & Garrod, S.
(2004) Toward a mechanistic psychology of dialogue. Behavioral and Brain Sciences, 27 , 169–226. DOI logoGoogle Scholar
Pinar Sanz, M.
(Ed.) (2015) Multimodality and cognitive linguistics. Amsterdam/Philadelphia: John Benjamins. DOI logoGoogle Scholar
Provine, R. R.
(2000) Laugher: A scientific investigation. New York: Viking.Google Scholar
Ritchie, G.
(2006) Reinterpretation and viewpoints. Humor, 19 (3), 251–270. DOI logoGoogle Scholar
Rossano, F.
(2012) Gaze in conversation. In J. Sidnell, & T. Stivers (Eds.), The handbook of Conversation Analysis (pp. 308–329). Chichester: Wiley-Blackwell. DOI logoGoogle Scholar
Selting, M., Auer, P., Barden, B., Couper-Kuhlen, E., Günther, S., Quasthoff, U., Schlobinski, P., & Uhmann, S.
(1998) Gesprachsanalytisches Transkriptionssystem (GAT). Linguistische Berichte, 173 , 91–122.Google Scholar
Selting, M.
(2000) The construction of units in conversational talk. Language in Society, 29 , 477–517. DOI logoGoogle Scholar
Semino, E., & Demjén, Z.
(Eds.) (2017) The Routledge handbook of metaphor and language. London: Routledge.Google Scholar
Sidnell, J.
(2006) Coordinating gesture, talk, and gaze in reenactments. Research on Language and Social Interaction, 39 (4), 377–409. DOI logoGoogle Scholar
Sperber, D., & Wilson, D.
(1981) Irony and the use-mention distinction. In P. Cole (Ed.), Radical pragmatics (pp. 295–318). New York: Academic Press.Google Scholar
(1995) Relevance: Communication and cognition. Oxford: Blackwell.Google Scholar
Sweetser, E., & Stec, K.
(2016) Maintaining multiple viewpoints with gaze. In B. Dancygier, W. Lu, & A. Verhagen (Eds.), Viewpoint and the fabric of meaning (pp. 237–258). Berlin: Mouton de Gruyter. DOI logoGoogle Scholar
Tabacaru, S., & Lemmens, M.
(2014) Facial expressions as gestural triggers in interactional humor. European Journal of Humour Research, 2 (2), 11–31. DOI logoGoogle Scholar
Tannen, D.
(1984) Conversational style. Analyzing talk among friends. Norwood, NJ: Ablex Publishing Corporation.Google Scholar
Thompson, S. A., & Suzuki, R.
(2014) Reenactments in conversation: Gaze and recipiency. Discourse Studies, 16 (6), 816–846. DOI logoGoogle Scholar
Tobin, V.
(2016) Performance, irony, and viewpoint in language. In A. Cook, & R. Blair (Eds.), Theatre, performance and cognition: Languages, bodies and ecologies (pp. 54–67). London: Bloomsbury Methuen. DOI logoGoogle Scholar
Tobin, V., & Israel, M.
(2012) Irony as a viewpoint phenomenon. In B. Dancygier, & E. Sweetser (Eds.), Viewpoint in language. A multimodal perspective (pp. 25–46). Cambridge: Cambridge University Press. DOI logoGoogle Scholar
Van Krieken, K., & Sanders, J.
(2019) Smoothly moving through mental spaces: Linguistic patterns of viewpoint transfer in news narratives. Cognitive Linguistics, 30 (3), 499–530. DOI logoGoogle Scholar
Vertegaal, R.
(1999) The GAZE Groupware system: Mediating joint attention in multiparty communication and collaboration. In M. G. Williams, & M. W. Altom (Eds.) Proceedings of CHI’99 (pp. 294–301). Pittsburgh: ACM. DOI logoGoogle Scholar
Zima, E., Weiß, C., & Brône, G.
(2018) Gaze and overlap resolution in triadic interactions. Journal of Pragmatics, 140 , 49–69. DOI logoGoogle Scholar
Zlatev, J.
(2019) Mimesis theory, learning and polysemiotic communication. In M. Peters (Ed.), Encyclopedia of educational philosophy and theory. Dordrecht: Springer. DOI logoGoogle Scholar
Cited by

Cited by 6 other publications

Attardo, Salvatore
2023. Tracking the Ironical Eye: Eye Tracking Studies on Irony and Sarcasm. In The Cambridge Handbook of Irony and Thought, ► pp. 140 ff. DOI logo
de Vries, Clarissa, Bert Oben & Geert Brône
2021. Exploring the role of the body in communicating ironic stance. Languages and Modalities 1 ► pp. 65 ff. DOI logo
Ladilova, Anna & Ulrike Schröder
2022. Humor in intercultural interaction: A source for misunderstanding or a common ground builder? A multimodal analysis. Intercultural Pragmatics 19:1 ► pp. 71 ff. DOI logo
Tasnia, Radiathun, Nabila Ayman, Afrin Sultana, Abu Nowshed Chy & Masaki Aono
2023. Exploiting stacked embeddings with LSTM for multilingual humor and irony detection. Social Network Analysis and Mining 13:1 DOI logo
[no author supplied]
2022. The Multimodal Performance of Conversational Humor [Figurative Thought and Language, 13], DOI logo
[no author supplied]
2023. Irony in Linguistic Communication. In The Cambridge Handbook of Irony and Thought, ► pp. 129 ff. DOI logo

This list is based on CrossRef data as of 20 march 2024. Please note that it may not be complete. Sources presented here have been supplied by the respective publishers. Any errors therein should be reported to them.