Cover not available
Part of
Mobile Eyetracking: New avenues for the study of gaze in social interaction
Edited by Elisabeth Zima and Anja Stukenbrock
[Pragmatics & Beyond New Series 351] 2025
► pp. 189–207

Chapter 7
Pupil size indicates planning effort at turn transitions in natural conversation

 | Leibniz-Institut für Deutsche SpracheMannheim
 | Leibniz-Institut für Deutsche SpracheMannheim
The study investigates the cognitive demands of speech planning in unrestricted, natural conversation. Focusing on question-answer sequences in triadic interactions, we analyse whether answerers, compared to not-answerers, exhibit increased cognitive effort during turn transitions. Using pupil size data from the Freiburg Multimodal Interaction Corpus, we find that answerers indeed show greater pupil dilation than not-answerers, suggesting heightened processing load during speech planning at transition-relevance places. This finding supports the hypothesis that speech planning is a primary contributor to increased cognitive effort during turn transitions, highlighting the value of pupillometry in the study of naturalistic conversation. The findings offer insights into the cognitive dynamics of multiparty social interaction, bridging the gap between controlled experiments and ecologically valid conversational settings.
Keywords: interaction, conversation, triads, turn-taking, turn-transitions, pupillometry, processing load, speech planning, question-answer sequences, corpus analysis
Article outline
  • 1.Introduction
  • 2.Methods
    • 2.1Data collection
    • 2.2Data Pre-processing and statistical analysis
  • 3.Results
  • 4.Discussion
  • 5.Conclusion
  • Author queries
  • Notes
  • References
  • Appendix
This content is being prepared for publication; it may be subject to changes.
References (66)
References
. 2021b. “Turn-Allocation and Gaze: A Multimodal Revision of the ‘Current-Speaker-Selects-next’ Rule of the Turn-Taking System of Conversation Analysis.” Discourse Studies 23 (2): 117–40. DOI logoGoogle Scholar
Barthel, Mathias. 2020. “Speech Planning in Dialogue — Psycholinguistic Studies of the Timing of Turn Taking.” PhD Thesis, Nijmegen: Radboud University Nijmegen. [URL]
. 2021. “Speech Planning Interferes with Language Comprehension: Evidence from Semantic Illusions in Question-Response Sequences.” In Proceedings of the 25th Workshop on the Semantics and Pragmatics of Dialogue, 1–14. Potsdam, Germany.Google Scholar
Barthel, Mathias, and Stephen C. Levinson. 2020. “Next Speakers Plan Word Forms in Overlap with the Incoming Turn: Evidence from Gaze-Contingent Switch Task Performance.” Language, Cognition and Neuroscience 35 (9): 1183–1202. DOI logoGoogle Scholar
Barthel, Mathias, Antje S. Meyer, and Stephen C. Levinson. 2017. “Next Speakers Plan Their Turn Early and Speak after Turn-Final ‘Go-Signals.’” Frontiers in Psychology 8: 393. DOI logoGoogle Scholar
Barthel, Mathias, and Sebastian Sauppe. 2019. “Speech Planning at Turn Transitions in Dialog Is Associated with Increased Processing Load.” Cognitive Science 43 (7): e12768. DOI logoGoogle Scholar
Barthel, Mathias, Sebastian Sauppe, Stephen C. Levinson, and Antje S. Meyer. 2016. “The Timing of Utterance Planning in Task-Oriented Dialogue: Evidence from a Novel List-Completion Paradigm.” Frontiers in Psychology 7: 1858. DOI logoGoogle Scholar
Bates, Douglas, Martin Mächler, Ben Bolker, and Steve Walker. 2015. “Fitting Linear Mixed-Effects Models Using Lme4.” Journal of Statistical Software 67 (1). DOI logoGoogle Scholar
Beatty, Jackson. 1982. “Task-Evoked Pupillary Responses, Processing Load, and the Structure of Processing Resources.” Psychological Bulletin 91 (2): 276–92. DOI logoGoogle Scholar
Boersma, Paul, and David Weenink. 2015. Praat: Doing Phonetics by Computer [Computer Program]. Version 5.3.56, [URL]
Bögels, Sara. 2020. “Neural Correlates of Turn-Taking in the Wild: Response Planning Starts Early in Free Interviews.” Cognition 203 (October): 104347. DOI logoGoogle Scholar
Bögels, Sara, and Stephen C. Levinson. 2017. “The Brain Behind the Response: Insights Into Turn-Taking in Conversation From Neuroimaging.” Research on Language and Social Interaction 50 (1): 71–89. DOI logoGoogle Scholar
Bögels, Sara, Lilla Magyari, and Stephen C. Levinson. 2015. “Neural Signatures of Response Planning Occur Midway through an Incoming Question in Conversation.” Scientific Reports 5 (12881): 1–11. DOI logoGoogle Scholar
Boiteau, Timothy W., Patrick S. Malone, Sara A. Peters, and Amit Almor. 2014. “Interference between Conversation and a Concurrent Visuomotor Task.” Journal of Experimental Psychology: General 143 (1): 295–311. DOI logoGoogle Scholar
Bradley, Margaret M., Laura Miccoli, Miguel A. Escrig, and Peter J. Lang. 2008. “The Pupil as a Measure of Emotional Arousal and Autonomic Activation.” Psychophysiology 45 (4): 602–7. DOI logoGoogle Scholar
Drews, Frank A., Monisha Pasupathi, and David L. Strayer. 2008. “Passenger and Cell Phone Conversations in Simulated Driving.” Journal of Experimental Psychology: Applied 14 (4): 392–400. DOI logoGoogle Scholar
Engelhardt, Paul E., Fernanda Ferreira, and Elena G. Patsenko. 2010. “Pupillometry Reveals Processing Load during Spoken Language Comprehension.” Quarterly Journal of Experimental Psychology 63 (4): 639–45. DOI logoGoogle Scholar
Ford, Cecilia E., Barbara A. Fox, and Sandra A. Thompson. 1996. “Practices in the Construction of Turns: The ‘TCU’ Revisited.” Pragmatics 6 (3): 427–54.Google Scholar
Ford, Cecilia E., and Sandra A. Thompson. 1996. “Interactional Units in Conversation: Syntactic, Intonational, and Pragmatic Recources for the Management of Turns.” In Interaction and Grammar, edited by Elinor Ochs, Emanuel A. Schegloff, and Sandra A. Thompson, 134–84. Cambridge: Cambridge University Press. DOI logoGoogle Scholar
Fox Tree, Jean E. 2002. “Interpreting Pauses and Ums at Turn Exchanges.” Discourse Processes 34 (1): 37–55. DOI logoGoogle Scholar
Hayashi, Makoto. 2012. “Turn Allocation and Turn Sharing.” In The Handbook of Conversation Analysis, edited by Jack Sidnell and Tanya Stivers, 1st ed., 167–90. Wiley. DOI logoGoogle Scholar
Heldner, Mattias. 2011. “Detection Thresholds for Gaps, Overlaps, and No-Gap-No-Overlaps.” The Journal of the Acoustical Society of America 130 (1): 508–13. DOI logoGoogle Scholar
Heldner, Mattias, and Jens Edlund. 2010. “Pauses, Gaps and Overlaps in Conversations.” Journal of Phonetics 38 (4): 555–68. DOI logoGoogle Scholar
Henetz, Tania. 2017. “Don’t Hesitate! The Length of Inter-Turn Gaps Influences Observers’ Interactional Attributions.” PhD Thesis, Stanford: Stanford University.
Heritage, John. 1984. Garfinkel and Ethnomethodology. Repr. Cambridge: Polity Press.Google Scholar
Hess, E. H., and J. M. Polt. 1964. “Pupil Size in Relation to Mental Activity during Simple Problem-Solving.” Science 143 (3611): 1190–92. DOI logoGoogle Scholar
Holler, Judith, Phillip M. Alday, Caitlin Decuyper, Mareike Geiger, Kobin H. Kendrick, and Antje S. Meyer. 2021. “Competition Reduces Response Times in Multiparty Conversation.” Frontiers in Psychology 12: 693124. DOI logoGoogle Scholar
Indefrey, Peter, and Willem J. M. Levelt. 2004. “The Spatial and Temporal Signatures of Word Production Components.” Cognition 92 (1–2): 101–44. DOI logoGoogle Scholar
Jefferson, Gail. 2004. “Glossary of Transcript Symbols with an Introduction.” In Pragmatics & Beyond New Series, edited by Gene H. Lerner, 125:13–31. Amsterdam: John Benjamins Publishing Company. DOI logoGoogle Scholar
Jescheniak, Jörg D., and Willem J. M. Levelt. 1994. “Word Frequency Effects in Speech Production: Retrieval of Syntactic Information and of Phonological Form.” Journal of Experimental Psychology: Learning, Memory, and Cognition 20 (4): 824–43. DOI logoGoogle Scholar
Just, Marcel A., and Patricia A. Carpenter. 1993. “The Intensity Dimension of Thought: Pupillometric Indices of Sentence Processing.” Canadian Journal of Experimental Psychology 47 (2): 310–39. DOI logoGoogle Scholar
Kahneman, Daniel, and Jackson Beatty. 1966. “Pupil Diameter and Load on Memory.” Science 154 (3756): 1583–85. DOI logoGoogle Scholar
Kendon, Adam. 1967. “Some Functions of Gaze-Direction in Social Interaction.” Acta Psychologica 26 (January): 22–63. DOI logoGoogle Scholar
Koch, Xaver, and Esther Janse. 2016. “Speech Rate Effects on the Processing of Conversational Speech across the Adult Life Span.” The Journal of the Acoustical Society of America 139 (4): 1618–36. DOI logoGoogle Scholar
Kubose, Tate T., Kathryn Bock, Gary S. Dell, Susan M. Garnsey, Arthur F. Kramer, and Jeff Mayhugh. 2006. “The Effects of Speech Production and Speech Comprehension on Simulated Driving Performance.” Applied Cognitive Psychology 20 (1): 43–63. DOI logoGoogle Scholar
Kuznetsova, Alexandra, Per B. Brockhoff, and Rune H. B. Christensen. 2017. “LmerTest Package: Tests in Linear Mixed Effects Models.” Journal of Statistical Software 82 (13). DOI logoGoogle Scholar
Laeng, Bruno, Sylvain Sirois, and Gustaf Gredebäck. 2012. “Pupillometry: A Window to the Preconscious?” Perspectives on Psychological Science 7 (1): 18–27. DOI logoGoogle Scholar
Lerner, Gene H. 2003. “Selecting next Speaker: The Context-Sensitive Operation of a Context-Free Organization.” Language in Society 32 (2): 177–201. DOI logoGoogle Scholar
Levelt, Willem J. M., Ardi Roelofs, and Antje S. Meyer. 1999. “A Theory of Lexical Access in Speech Production.” Behavioural and Brain Sciences 22 (01): 1–75. DOI logoGoogle Scholar
Levinson, Stephen C., and Francisco Torreira. 2015. “Timing in Turn-Taking and Its Implications for Processing Models of Language.” Frontiers in Psychology 6 (731): 10–26. DOI logoGoogle Scholar
Mathôt, Sebastiaan. 2018. “Pupillometry: Psychology, Physiology, and Function.” Journal of Cognition 1 (1): 16. DOI logoGoogle Scholar
Papesh, Megan H., and Stephen D. Goldinger. 2012. “Pupil-BLAH-Metry: Cognitive Effort in Speech Planning Reflected by Pupil Dilation.” Attention, Perception, & Psychophysics 74 (4): 754–65. DOI logoGoogle Scholar
Roberts, Felicia, and Alexander L. Francis. 2013. “Identifying a Temporal Threshold of Tolerance for Silent Gaps after Requests.” The Journal of the Acoustical Society of America 133 (6): EL471–77. DOI logoGoogle Scholar
Roberts, Felicia, Alexander L. Francis, and Melanie Morgan. 2006. “The Interaction of Inter-Turn Silence with Prosodic Cues in Listener Perceptions of ‘Trouble’ in Conversation.” Speech Communication 48 (9): 1079–93. DOI logoGoogle Scholar
Roberts, Seán G., Francisco Torreira, and Stephen C. Levinson. 2015. “The Effects of Processing and Sequence Organization on the Timing of Turn Taking: A Corpus Study.” Frontiers in Psychology 6. DOI logoGoogle Scholar
Rühlemann, Christoph. 2024. “Gaze alternation predicts inclusive next-speaker selection: Evidence from eyetracking.” Frontiers in Communication (Sec. Multimodality of Communication) 9:1396925. DOI logoGoogle Scholar
Rühlemann, Christoph, and Mathias Barthel. 2024. “Word frequency and cognitive effort in turns-at-talk: Turn structure affects processing load in natural conversation.” Frontiers in Psychology (Sec. Psychology of Language) 15. DOI logoGoogle Scholar
. Under revision. “Speech planning depends on next-speaker selection: Evidence from pupillometry in question-answer sequences in naturalistic triadic conversation.”
Rühlemann, Christoph, and Alexander Ptak. 2023. “Reaching beneath the Tip of the Iceberg: A Guide to the Freiburg Multimodal Interaction Corpus.” Open Linguistics 9 (1). DOI logoGoogle Scholar
Sacks, Harvey, Emanuel A. Schegloff, and Gail Jefferson. 1974. “A Simplest Systematics for the Organization of Turn-Taking for Conversation.” Language 50 (4): 696–735. DOI logoGoogle Scholar
Sauppe, Sebastian. 2017. “Symmetrical and Asymmetrical Voice Systems and Processing Load: Pupillometric Evidence from Sentence Production in Tagalog and German.” Language 93 (2): 288–313. DOI logoGoogle Scholar
Schmidtke, Jens. 2014. “Second Language Experience Modulates Word Retrieval Effort in Bilinguals: Evidence from Pupillometry.” Frontiers in Psychology 5: 137. DOI logoGoogle Scholar
Selting, Margret. 2000. “The Construction of Units in Conversational Talk.” Language in Society 29: 477–517. DOI logoGoogle Scholar
Sevilla, Yamila, Mora Maldonado, and Diego E. Shalóm. 2014. “Pupillary Dynamics Reveal Computational Cost in Sentence Planning.” Quarterly Journal of Experimental Psychology 67 (6): 1041–52. DOI logoGoogle Scholar
Sirois, Sylvain, and Julie Brisson. 2014. “Pupillometry.” Wiley Interdisciplinary Reviews: Cognitive Science 5 (6): 679–92. DOI logoGoogle Scholar
Stivers, Tanja, Nick J. Enfield, Penelope Brown, C. Englert, Makoto Hayashi, Trine Heinemann, Gertie Hoymann, et al. 2009. “Universals and Cultural Variation in Turn-Taking in Conversation.” Proceedings of the National Academy of Sciences 106 (26): 10587–92. DOI logoGoogle Scholar
Stivers, Tanya, and Federico Rossano. 2010. “Mobilizing Response.” Research on Language & Social Interaction 43 (1): 3–31. DOI logoGoogle Scholar
The Language Archive. 2023. ELAN (version 6.7). Nijmegen: Max Planck Institute for Psycholinguistics. [URL]
Tromp, Johanne, Peter Hagoort, and Antje S. Meyer. 2016. “Pupillometry Reveals Increased Pupil Size during Indirect Request Comprehension.” Quarterly Journal of Experimental Psychology 69 (6): 1093–1108. DOI logoGoogle Scholar
Walker, M. B., and C. Trimboli. 1984. “The Role of Nonverbal Signals in Co-Ordinating Speaking Turns.” Journal of Language and Social Psychology 3 (4): 257–72. DOI logoGoogle Scholar
Wel, Pauline van der, and Henk van Steenbergen. 2018. “Pupil Dilation as an Index of Effort in Cognitive Control Tasks: A Review.” Psychonomic Bulletin & Review 25 (6): 2005–15. DOI logoGoogle Scholar
Wu, Esther Xiu Wen, Bruno Laeng, and Svein Magnussen. 2012. “Through the Eyes of the Own-Race Bias: Eye-Tracking and Pupillometry during Face Recognition.” Social Neuroscience 7 (2): 202–16. DOI logoGoogle Scholar
Zekveld, Adriana A., Sophia E. Kramer, and Joost M. Festen. 2010. “Pupil Response as an Indication of Effortful Listening: The Influence of Sentence Intelligibility:” Ear and Hearing 31 (4): 480–90. DOI logoGoogle Scholar
Zima, Elisabeth, Clarissa Weiß, and Geert Brône. 2019. “Gaze and Overlap Resolution in Triadic Interactions.” Journal of Pragmatics 140 (January): 49–69. DOI logoGoogle Scholar