Cover not available
Part of
Time in Embodied Interaction: Synchronicity and sequentiality of multimodal resources
Edited by Arnulf Deppermann and Jürgen Streeck
[Pragmatics & Beyond New Series 293] 2018
► pp. 31–68

Chapter 1
Forward-looking

Where do we go with multimodal projections?

This chapter deals with the concepts of projection and conditional relevance by examining the temporal and sequential relationship between speech, gesture and gaze in deictic practices. It makes the following claims: Verbal deictics combined with embodied practices that direct the addressee’s attention to visible phenomena form a multimodal, intercorporeal gestalt. They constitute a request for the addressee’s gaze. This means that within the deictic referencing act, a multimodal summons-answer sequence unfolds that comprises recurrent gaze patterns both of the pointing and the gazing participant. These gaze practices are context-sensitive and temporally fine-tuned to the emerging verbal and kinetic practices, they occupy specific positions within the deictic act which in turn can be embedded within other actions such as instructions, requests etc.
Keywords: projection, conditional relevance, gaze, multimodal gestalt, deixis
Article outline
  • 1.Introduction
  • 2.Multimodal projection and conditional relevance across modalities and actions
  • 3.Multimodal projection within utterances and actions
  • 4.Projections between speech, gaze and gesture in deictic utterances
    • 4.1 Summoning the addressee’s gaze
    • 4.2 Monitoring the addressee’s gaze
    • 4.3 Repair after gaze monitoring
    • 4.4 Gaze in deictic summons-answer sequences: Evidence from dual mobile eye-tracking
  • 5. Conclusion
  • Acknowledgements
  • Notes
  • References
  • Appendix
Published online: 13 September 2018
DOI logo
https://doi.org/10.1075/pbns.293.01stu
References
Auer, Peter
2005 “Projection in Interaction and Projection in Grammar.” Text and Talk 25 (1): 7–36.Google Scholar
2015 “The Temporality of Language in Interaction: Projection and Latency.” In Temporality in Interaction, ed. by Deppermann, Arnulf, and Susanne Günthner, 27–56. Amsterdam: John Benjamins.Google Scholar
Brooks, Rechele, and Andrew N. Meltzoff
2014 “Gaze Following: A Mechanism for Building Social Connections between Infants and Adults.” In Mechanisms of Social Connection: From Brain to Group, ed. by Mikulincer, Mario, and Phillip R. Shave, 167–183. Washington, D.C.: American Psychological Association.DOI logoGoogle Scholar
Bühler, Karl
2011[1934]. Theory of Language. The Representational Function of Language. Translated by D. F. Goodwin in collaboration with A. Eschbach. Amsterdam: John Benjamins.DOI logoGoogle Scholar
Couper-Kuhlen, Elizabeth, and Dagmar Barth-Weingarten
2011 “A system for transcribing talk-in-interaction: GAT 2 translated and adapted for English.” Gesprächsforschung – Onlinezeitschrift zur verbalen Interaktion 12, 1–51. Retrieved from [URL].Google Scholar
Dausenschön-Gay, Ulrich, and Ulrich Krafft
2009 “Preparing Next Actions in Routine Activities.” Discourse Processes 46: 247–268.DOI logoGoogle Scholar
Deppermann, Arnulf
2015 “When Recipient Design Fails: Egocentric Turn-Design of Instructions in Driving Lessons Leading to Breakdowns of Intersubjectivity.” Gesprächsforschung – Online-Zeitschrift zur verbalen Interaktion 16: 63–101.Google Scholar
Deppermann, Arnulf, and Susanne Günthner
(eds) 2015 Temporality in Interaction. Philadelphia: John Benjamins.DOI logoGoogle Scholar
Deppermann, Arnulf, and Reinhold Schmitt
2007 “Koordination. zur Begründung eines neuen Forschungsgegenstandes.” In Koordination. Analysen zur multimodalen Interaktion, ed. by Schmitt, Reinhold, 15–54. Tübingen: Narr.Google Scholar
De Stefani, Elwys, and Anne-Daniele Gazin
2014 “Instructional Sequences in Driving Lessons: Mobile Participants and the Temporal and Sequential Organization of Actions.” Journal of Pragmatics 65: 63–79.DOI logoGoogle Scholar
Duchowski, Andrew T.
2003 Eye Tracking Methodology. Theory and Practice. Berlin: Springer.DOI logoGoogle Scholar
Fillmore, Charles
1997 Lectures on Deixis. Stanford, CA: CSLI.Google Scholar
Goffman, Erving
1963 Behavior in Public Places. Notes on the Social Organization of Gatherings. New York: The Free Press.Google Scholar
Goodwin, Charles
1980 “Restarts, Pauses, and the Achievement of a State of Mutual Gaze at Turn-Beginning.” Sociological Inquiry 50 (3–4): 272–302.DOI logoGoogle Scholar
1994 “Professional Vision.” American Anthropologist 96 (3): 606–633.DOI logoGoogle Scholar
2000 “Action and Embodiment within Situated Human Interaction.” Journal of Pragmatics 32: 1489–1522.DOI logoGoogle Scholar
2003 “Pointing as Situated Practice.” In Pointing. Where Language, Culture, and Cognition Meet, ed. by Kita, Sotaro, 217–241. NJ: Erlbaum.Google Scholar
2007 “Interactive Footing.” In Reporting Talk. Reported Speech in Interaction, ed. by Holt, Elizabeth, and Rebecca Clift, 16–46. Cambridge: Cambridge University Press.Google Scholar
Goodwin, Charles, and Marjorie H. Goodwin
1987 “Concurrent Operations on Talk. Notes on the Interactive Organization of Assessments.” IPrA Papers in Pragmatics 1 (1): 1–54.DOI logoGoogle Scholar
Goodwin, Marjorie H.
2006 “Participation, Affect, and Trajectory in Family Directive/Response Sequences.” Text and Talk 26: 513–541.Google Scholar
Hausendorf, Heiko
2003 “Deixis and Speech Situation Revisited. The Mechanism of Perceived Perception.” In Deictic Conceptualisation of Space, Time and Person, ed. by Lenz, Friedrich, 249–269. Amsterdam: John Benjamins.DOI logoGoogle Scholar
Heath, Christian
1986 Body Movement and Speech in Medical Interaction. Cambridge: Cambridge University Press.DOI logoGoogle Scholar
Keevallik, Leelo
2013 “Here in Time and Space: Decomposing Movement in Dance Instruction.” In Interaction and Mobility. Language and the Body in Motion, ed. by Haddington, Pentti, Mondada, Lorenza, and Maurice Nevile, 345–370. Berlin: De Gruyter.DOI logoGoogle Scholar
Kendon, Adam
1990 Conducting Interaction. Patterns of Behavior in Focused Encounters. Cambridge: Cambridge University Press.Google Scholar
2004 Gesture. Visible Action as Utterance. Cambridge: Cambridge University Press.DOI logoGoogle Scholar
Kita, Sotaro
(ed) 2003 Pointing. Where Language, Culture, and Cognition Meet. Mahwah, NJ: Erlbaum.Google Scholar
Land, Michael F.
2006 “Eye Movement and the Control of Actions in Everyday Life.” Progress in Retinal and Eye Research 25: 296–324.DOI logoGoogle Scholar
Levinson, Stephen C.
2013 “Action Ascription and Formation.” In The Handbook of Conversation Analysis, ed. by Sidnell, Jack, and Tanya Stivers, 103–130. Wiley: Blackwell.Google Scholar
Lindwall, Oskar, and Anna Ekström
2012 “Instruction-in-Interaction: The Teaching and Learning of a Manual Skill.” Human Studies 35: 27–49.DOI logoGoogle Scholar
Lindwall, Oskar, Lymer, Gustav, and Christian Greiffenhagen
2015 “The Sequential Analysis of Instruction.” In The Handbook of Classroom Discourse and Interaction, ed. by Markee, Numa 142–157. Oxford: Wiley and Blackwell.Google Scholar
Meyer, Christian, Streeck, Jürgen, and Scott Jordan
eds to appear Intercorporeality. Emerging Socialities in Interaction Oxford Oxford University Press
Mondada, Lorenza
2009 “The Embodied and Negotiated Production of Assessments in Instructed Actions.” Research on Language and Social Interaction 42 (4): 329–361.DOI logoGoogle Scholar
2014 “The Local Constitution of Multimodal Resources for Social Interaction.” Journal of Pragmatics 65: 137–156.DOI logoGoogle Scholar
2015 “Multimodal Completions.” In Temporality in Interaction, ed. by Deppermann, Arnulf, and Susanne Günthner, 267–307. Amsterdam: John Benjamins.Google Scholar
Rauniomaa, Mira, and Tiina Keisanen
2012 “Two multimodal formats for responding to requests.” Journal of Pragmatics 44, 829–842.Google Scholar
Rossano, Federico
2012 Gaze Behavior in Face-to-Face Interaction. Nijmegen: Max Planck Institute for Psycholinguistics Series.Google Scholar
Sacks, Harvey, Emanuel A. Schegloff, and Gail Jefferson
1974 “A Simplest Systematics for the Organization of Turn-Taking for Conversation.” Language 50(4), 696–735.Google Scholar
Sacks, Harvey
1972 “On the Analyzability of Stories by Children.” In Directions in Sociolinguistics. The Ethnography of Speaking, ed. by Gumperz, John J., and Dell Hymes, 329–345. New York: Holt, Rinehart and Winston.Google Scholar
1984 “Notes on Methodology.” In Structures of Social Action. Studies in Conversation Analysis, ed. by Atkinson, John M., and John C. Heritage, 21–27. Cambridge: Cambridge University Press.Google Scholar
Schegloff, Emanuel A.
1968 “Sequencing in Conversational Openings.” American Anthropologist 70 (6): 1075–1095.DOI logoGoogle Scholar
1992 “In Another Context.” In Rethinking Context. Language as an Interpretative Phenomenon, ed. by Goodwin, Charles, and Alessandro Duranti, 191–228. Cambridge: Cambridge University Press.Google Scholar
2007 Sequence Organization in Interaction. Cambridge: Cambridge University Press.DOI logoGoogle Scholar
Schegloff, Emanuel A., and Harvey Sacks
1973 “Opening Up Closings.” Semiotica 8: 289–327.DOI logoGoogle Scholar
Selting, Margret
et al. (2009) “Gesprächsanalytisches Transkriptionssystem 2 (GAT 2).” Gesprächsforschung – Onlinezeitschrift zur verbalen Interaktion 10, 353–402. Retrieved from [URL].Google Scholar
Stivers, Tanya, and Federico Rossano
2010 “A Scalar View of Response Relevance.” Research on Language and Social Interaction 43(1), 49–54.Google Scholar
Streeck, Jürgen
1988 “The Significance of Gesture: How it is Established.” IPrA Papers in Pragmatics 2(1-2), 60–83.Google Scholar
1993 “Gesture as Communication I: Its Coordination with Gaze and Speech.” Communication Monographs 60 (4): 275–299.DOI logoGoogle Scholar
1995 “On Projection.” In Social Intelligence and Interaction. Expressions and Implications of Social Bias in Human Intelligence, ed. by Goody, Esther N., 87–110. Cambridge: Cambridge University Press.DOI logoGoogle Scholar
2002 “Grammars, Words, and Embodied Meanings: On the Uses and Evolution of So and Like.” Journal of Communication 52 (3): 581–596.DOI logoGoogle Scholar
2007 “Geste und verstreichende Zeit: Innehalten und Bedeutungswandel der “bietenden Hand.” In Gespräch als Prozess. Linguistische Aspekte der Zeitlichkeit verbaler Interaktion, ed. by Hausendorf, Heiko, 157–180. Tübingen: Narr.Google Scholar
2009 Gesturecraft. The Manu-Facture of Meaning. Amsterdam: John Benjamins.DOI logoGoogle Scholar
Streeck, Jürgen, and Ulrike Hartge
1992 “Previews: Gestures at the Transition Place.” In The Contextualization of Language, ed. by Auer, Peter, and Aldo Di Luzio, 135–158. Amsterdam: Benjamins.DOI logoGoogle Scholar
Stukenbrock, Anja
2008 ““Wo ist der Hauptschmerz?” – Zeigen am eigenen Körper in der medizinischen Kommunikation.” Gesprächsforschung – Online-Zeitschrift zur verbalen Interaktion 9: 1–33.Google Scholar
2009 “Referenz durch Zeigen: Zur Theorie der Deixis.” Deutsche Sprache 37: 289–315.Google Scholar
2010 “Überlegungen zu einem multimodalen Verständnis der gesprochenen Sprache am Beispiel deiktischer Verwendungsweisen des Ausdrucks so.” InLiSt – Interaction and Linguistic Structures 47 ([URL]).
2014a “Take the Words out of My Mouth: Verbal Instructions as Embodied Practices.” Journal of Pragmatics 65: 80–102.DOI logoGoogle Scholar
2014b “Pointing to an ‘Empty’ Space: Deixis am Phantasma in Face-to-Face Interaction.” Journal of Pragmatics 74: 70–93.DOI logoGoogle Scholar
2015a Deixis in der face-to-face-Interaktion. Berlin: De Gruyter.DOI logoGoogle Scholar
2015b “pi mal Daumen. Über eine kontextreiche Unschärferelation beim Zeigen.” In Sprache im Gebrauch: räumlich, zeitlich, interaktional, ed. by Bergmann, Pia, Birkner, Gilles, Peter, Spiekermann, Helmut, and Tobias Streck, 343–360. Heidelberg: Winter.Google Scholar
2016 “Deiktische Praktiken: Zwischen Interaktion und Grammatik.” In Sprachliche und kommunikative Praktiken. Jahrburch 2015 des Instituts für Deutsche Sprache, ed. by Deppermann, Arnulf, Feilke, Helmut, and Angelika Linke, 81–126. Berlin: De Gruyter.Google Scholar
2017 “Intercorporeal Phantasms: Kinesthetic Alignment with Imagined Bodies in Self-Defense Trainings.” In: Intercorporeality. Emerging Socialities in Interaction, ed. by Meyer, Christian, Streeck, Jürgen, and Scott Jordan, 237–263. Oxford: Oxford University Press.Google Scholar
in press. “Mobile Dual Eye-Tracking in Face-to-Face Interaction: The Case of Deixis and Joint Attention.” In Eye Tracking in Interaction. Studies on the Role of Eye Gaze in Dialogue ed. by Brône, Geert, and Bent Oben Amsterdam John Benjamins
Stukenbrock, Anja, and Karin Birkner
2012 “Multimodale Ressourcen bei Stadtführungen.” In Deutschland als fremde Kultur: Vermittlungsverfahren in Touristenführungen, ed. by Costa, Marcella, and Bernd Müller-Jacquier, 214–243. München: Judicium.Google Scholar
Svensson, Marcus S., Luff, Paul, and Christian Heath
2009 “Embedding Instruction in Practice: Contingency and Collaboration during Surgical Training.” Sociology of Health and Illness 31 (6): 889–906.DOI logoGoogle Scholar
Cited by

Cited by 18 other publications

Avgustis, Iuliia & Florence Oloff
2023. Getting (Others) Involved with Smartphones: Participation in Showing Sequences in Multiparty Settings. In Complexity of Interaction, ► pp. 297 ff. DOI logo
Choe, Ann Tai & Junichi Yagi
2023. Embodied remembering in coordinated performances. Multimodal Communication 12:2 ► pp. 99 ff. DOI logo
Deppermann, Arnulf & Alexandra Gubina
2021. When the Body Belies the Words: Embodied Agency With darf/kann ich? (“May/Can I?”) in German. Frontiers in Communication 6 DOI logo
Deppermann, Arnulf, Lorenza Mondada & Simona Pekarek Doehler
2021. Early Responses: An Introduction. Discourse Processes 58:4 ► pp. 293 ff. DOI logo
Deppermann, Arnulf & Axel Schmidt
2021. Micro-Sequential Coordination in Early Responses. Discourse Processes 58:4 ► pp. 372 ff. DOI logo
Dimitrova, Nevena
2020. The Role of Common Ground on Object Use in Shaping the Function of Infants’ Social Gaze. Frontiers in Psychology 11 DOI logo
Ehmer, Oliver
2021. Synchronization in demonstrations. Multimodal practices for instructing body knowledge. Linguistics Vanguard 7:s4 DOI logo
Ehmer, Oliver & Daniel Mandel
2021. Projecting action spaces. On the interactional relevance of cesural areas in co-enactments. Open Linguistics 7:1 ► pp. 638 ff. DOI logo
Krug, Maximilian
2022. Temporal procedures of mutual alignment and synchronization in collaborative meaning-making activities in a dance rehearsal. Frontiers in Communication 7 DOI logo
Li, Xiaoting
2021. Multimodal practices for negative assessments as delicate matters: Incomplete syntax, facial expressions, and head movements. Open Linguistics 7:1 ► pp. 549 ff. DOI logo
Norrthon, Stefan
2023. Cueing in Theatre: Timing and Temporal Variance in Rehearsals of Scene Transitions. Human Studies 46:2 ► pp. 199 ff. DOI logo
Salminen-Saari, Jessica F. A., Enrique Garcia Moreno-Esteva, Eeva Haataja, Miika Toivanen, Markku S. Hannula & Anu Laine
2021. Phases of collaborative mathematical problem solving and joint attention: a case study utilizing mobile gaze tracking. ZDM – Mathematics Education 53:4 ► pp. 771 ff. DOI logo
Schmidt, Axel & Arnulf Deppermann
2023. On the Emergence of Routines: An Interactional Micro-history of Rehearsing a Scene. Human Studies 46:2 ► pp. 273 ff. DOI logo
Singh, Ajit
2021. Situating embodied action plans: pre-enacting and planning actions within knowledge communication in sports training. Linguistics Vanguard 7:s4 DOI logo
Stukenbrock, Anja
2020. Deixis, Meta-Perceptive Gaze Practices, and the Interactional Achievement of Joint Attention. Frontiers in Psychology 11 DOI logo
Stukenbrock, Anja & Anh Nhi Dao
2019. Joint Attention in Passing: What Dual Mobile Eye Tracking Reveals About Gaze in Coordinating Embodied Activities at a Market. In Embodied Activities in Face-to-face and Mediated Settings, ► pp. 177 ff. DOI logo
Tschacher, Wolfgang & Hermann Haken
2019. Outlook. In The Process of Psychotherapy, ► pp. 153 ff. DOI logo
Yagi, Junichi
2023. “Five” or “ten”: analysing a co-operative correction in Muay Thai coaching. Sports Coaching Review ► pp. 1 ff. DOI logo

This list is based on CrossRef data as of 18 november 2023. Please note that it may not be complete. Sources presented here have been supplied by the respective publishers. Any errors therein should be reported to them.