Chapter 1.Approaching the multimodal study of conversational humor
1
1.1Why a multimodal study of conversational humor?
1
1.2The field of humor studies
6
1.2.1The linguistics of humor
7
1.2.2Cognitive linguistics and humor
8
1.3The study of conversational humor
10
1.3.2Defining and identifying conversational humor
12
1.4The study of humor markers
14
1.4.1Terminological issues in the study of humor markers
15
1.4.2Prosodic cues of humor
17
1.4.2.1Empirical and experimental studies of prosodic humor cues
19
1.4.3Laughter as a humor cue
23
1.4.4Irony cues
25
1.5Scope and organization of the volume
28
Chapter 2.Performing conversational humor multimodally – an overview
31
2.1Introduction
31
2.2Researching the multimodal performance of conversational humor
32
2.2.1Data collection setting and instruments
32
2.2.2Participants
35
2.2.3Data collection procedure
36
2.2.4Treatment of data
38
2.3Individual smiling behavior
40
2.4Joint negotiation and co-construction of the humorous frame
43
2.5Smiling dialogic synergy
45
2.6Gaze
47
2.6.1Gaze aversion and conversational humor
48
2.6.2Gaze to the mouth and conversational irony
49
2.7Failed humor
51
2.8Conclusions
52
Chapter 3.Individual smiling behavior
55
3.1Introduction
55
3.2Smiling as a social emotional expression
55
3.2.1Smiling within the Basic Emotion Theory (BET)
56
3.2.2Smiling within the Behavioral Ecology View (BEV)
56
3.2.3Smiling within the Theory of Affective Pragmatics (TAP)
58
3.3The study of smiling intensity
59
3.3.1Facial Action Coding System (FACS)
60
3.3.2Smiling intensity scales
62
3.3.2.1Additive FACS-based scoring systems and applications
62
3.3.2.2Holistic FACS-based scoring system
63
3.3.2.2.1The Smiling Intensity Scale (SIS)
64
3.4Smiling and humor
67
3.4.1Duchenne display and phony smiles
67
3.4.2Smiling voice
70
3.5Individual smiling behavior and conversational humor
73
3.5.1Smiling intensity during humorous and non-humorous discourse
73
3.5.2Smiling intensity in English and Spanish
78
3.5.3Smiling intensity by males and females
79
3.5.4Smiling intensity across types of humor
81
3.5.5Smiling intensity across conversational roles
82
3.6Conclusions
84
Chapter 4.Smiling patterns and dialogical smiling synergy
87
4.1Introduction
87
4.2Social cognitive theories of interactional behavior
87
4.2.1Behavioral alignment, synchron(icit)y, and coupling
88
4.2.2From alignment to synergy
91
4.2.3Embodied cognition and facial displays
93
4.3Analyzing smiling patterns and synchronicity during humorous discourse
94
4.4Smiling patterns during humorous discourse
95
4.4.1Framing smiling pattern
97
4.4.2Accommodation gesture
99
4.4.3Deadpan deliveries of humor
100
4.4.4Peak smiling pattern
103
4.5Smiling synchronicity
105
4.6Conclusions
109
Chapter 5.Eye movements
115
5.1Introduction
115
5.2The study of eye movements
115
5.2.1Brief history of eye-tracking
116
5.2.2How people look at faces
117
5.2.3Social eye-tracking
119
5.3Social, affective, and cognitive functions of gaze
123
5.3.1The processing of written irony
125
5.3.2Eye movements and humor
127
5.4Social eye-tracking for conversational humor
128
5.4.1Gaze to the interlocutor’s smiling facial expression: Eyes and mouth
128
5.4.2Gaze to the interlocutor’s eyes or mouth
130
5.4.3The effect of conversational role on gaze
133
5.5Linear mixed model analyses
135
5.5.1Gaze duration and facial smiling expression (eyes and mouth considered together)
136
5.5.2Duration of gaze to the interlocutor’s mouth or eyes
139
5.5.3Fixation duration and humor
140
5.6Conclusions
141
Chapter 6.Failed conversational humor
145
6.1Introduction
145
6.2What is failed humor?
145
6.2.1Models and empirical studies on failed humor
147
6.2.2Failed humor: Cues, signals, and responses
150
6.3Multimodal cues of failed conversational humor
152
6.3.1Failed humor understanding
153
6.3.2Failed humor appreciation or agreement
159
6.3.3Joint failed humor appreciation
161
6.4Conclusions
164
Chapter 7.Conclusions: Looking backwards and looking forward
169
7.1Introduction
169
7.2How conversational humor is performed multimodally
171
7.2.1The multimodal performance of successful punchlines
171
7.2.2The multimodal performance of successful jablines
172
7.2.3The multimodal performance of successful ironic comments
173
7.2.4The multimodal performance of failed humor
174
7.3Implications for applied humor research
175
7.3.1How smiling supports the negotiation of the humorous framing
176
7.3.2How gaze supports the negotiation of the humorous framing
177
7.4Limitations and future research directions
179
7.4.1Towards a multimodal theory of humor performance
182