Cover not available
Article published In:
Gesture
Vol. 17:3 (2018) ► pp.417–463

The multidimensionality of pointing

| Humboldt-Universität zu Berlin, Berlin School of Mind and Brain
| Universidade de São Paulo

This paper proposes a novel analysis of deictic gestures which yields a taxonomy of manual pointing. ‘Gesture form analysis’ brings into relief the diversity of pointing by considering the imaginary forms necessarily involved in interpreting a gesture. It combines into a single framework insights found in the literature on how the meaning of any gesture is enabled by a series of spatial operations leading from the physical form of the articulators to the form of the target. Seven distinct spatial operations combine to define a gesture type, twenty-seven of which are illustrated with examples from open-data corpora. Most types involve not the prototypical linear vector of pointing, but the plane of an open hand. Not only deictic, but also iconic and other functions are shown to be rooted in imaginary forms and their ability to draw attention to and specify locations, directions, areas and volumes of space.

Keywords: gesture form, imaginary forms, geometry, taxonomy, classification, deixis, iconicity
Article outline
  • Introduction
    • Gesture form analysis
    • Related literature
    • Structure of the empirical part of this paper
    • Open resources
  • Pointing with a 1D profile
    • Pointing At Object
    • Pointing At Location
    • Directing
    • Dynamic Directing
    • Drawing
    • Silhouetting
    • Preliminary conclusions
  • Pointing with a 2D profile
    • Group 1: Surface-edge marking
    • Group 2: Non-intersecting surface pointing
    • Group 3: Intersecting surface pointing
    • Group 4: Sketching with surface
    • Non-Specific Holding
  • Representing in 0D
  • Discussion
  • Acknowledgements
  • Notes
  • References
Published online: 18 February 2020
DOI logo
https://doi.org/10.1075/gest.17018.has
References
Andrén, Mats
(2010) Children’s gestures from 18 to 30 months. Doctoral Dissertation, Lund University.Google Scholar
Arnheim, Rudolf
(2004 [1969]) Visual thinking. Berkeley: University of California Press.Google Scholar
Bühler, Karl
(2000 [1934]) Sprachtheorie. In Ludger Hoffmann (Ed.), Sprachwissenschaft. Ein Reader (2nd Ed., pp. 51–71). Berlin: Walter de Gruyter.Google Scholar
Butterworth, George
(2003) Pointing is the royal road to language for babies. In Sotaro Kita (Ed.), Pointing: Where language, culture, and cognition meet (pp. 9–34). Mahwah, NJ: Lawrence Erlbaum.Google Scholar
Clark, Herbert H.
(1996) Using language. Cambridge: Cambridge University Press. DOI logoGoogle Scholar
Clark, Herbert H., Robert Schreuder, Samuel Buttrick
(1983) Common ground and the understanding of demonstrative reference. Journal of Verbal Learning and Verbal Behavior, 22 (2), 245–258. DOI logoGoogle Scholar
Cooperrider, Kensy A.
(2011) Reference in action: Links between pointing and language. PhD Dissertation, University of California San Diego.Google Scholar
(2014) Body-directed gestures: Pointing to the self and beyond. Journal of Pragmatics, 711, 1–16. DOI logoGoogle Scholar
Deacon, Terrence W.
(1997) The symbolic species: The co-evolution of language and the brain. New York & London: W. W. Norton and Company.Google Scholar
Enfield, Nick J.
(2001) Lip-pointing: A discussion of form and function with reference to data from Laos. Gesture, 1 (2), 185–211. DOI logoGoogle Scholar
(2009) The Anatomy of Meaning. Speech, gesture, and composite utterances. Cambridge & New York: Cambridge University Press. DOI logoGoogle Scholar
Fricke, Ellen
(2007) Origo, Geste und Raum. Lokaldeixis im Deutschen. Berlin/New York: Walter de Gruyter. DOI logoGoogle Scholar
Gibson, James J.
(1977) The theory of affordances. In Robert Shaw (Ed.), Perceiving, acting, and knowing. Towards an ecological psychology (pp. 67–82). Hillsdale, NJ: Lawrence Erlbaum.Google Scholar
Goodwin, Charles
(2003) Pointing as situated practice. In Sotaro Kita (Ed.), Pointing: Where language, culture, and cognition meet (pp. 217–242). Mahwah, NJ: Lawrence Erlbaum.Google Scholar
(2018) Co-operative action. Cambridge: Cambridge University Press.Google Scholar
Hassemer, Julius
(2009) Chief pointing-eye. Multiarticulatory metaphorical gestures expressing high. MA thesis, European University Viadrina, Frankfurt (Oder). DOI logoGoogle Scholar
(2016) Towards a theory of gesture form analysis. Imaginary forms as part of gesture conceptualisation, with empirical support from motion-capture data. Doctoral Dissertation, RWTH Aachen University.Google Scholar
Hassemer, Julius, Gina Joue, Klaus Willmes, & Irene Mittelberg
(2011) Dimensions and mechanisms of form constitution: Towards a formal description of gestures. In Proceedings of Gesture and Speech in Interaction (GESPIN) 2011. Bielefeld.Google Scholar
Hassemer, Julius, Cornelia Müller, & Irene Mittelberg
(2010) Marker analysis: Geometrical dimensions of gesture form. In 4th Conference of the International Society of Gesture Studies (ISGS4). July 25–30, Frankfurt (Oder).Google Scholar
Hassemer, Julius & Bodo Winter
(2016) Producing and perceiving gestures conveying height or shape. Gesture, 15 (3), 404–424. DOI logoGoogle Scholar
(2018) Decoding gestural iconicity. Cognitive Science, 42 (8), 3034–3049. DOI logoGoogle Scholar
Haviland, John B.
(2003) How to point in Zinacantán. In Sotaro Kita (Ed.), Pointing: Where language, culture, and cognition meet (pp. 139–170). Mahwah, NJ: Lawrence Erlbaum.Google Scholar
James, William
(1890) The principles of psychology, Vol. 21. New York: Henry Holt.Google Scholar
Kendon, Adam
(1988) Goffman’s approach to face-to-face Interaction. In Paul Drew & Anthony Wootton (Eds.), Erving Goffman. Exploring the interaction order (pp. 14–40). Boston: Northeastern University Press.Google Scholar
(2004) Gesture: Visible action as utterance. Cambridge & New York: Cambridge University Press. DOI logoGoogle Scholar
Kendon, Adam & Laura Versante
(2003) Pointing by hand in “Neapolitan”. In Sotaro Kita (Ed.), Pointing: Where language, culture, and cognition meet (pp. 109–138). Mahwah, NJ: Lawrence Erlbaum.Google Scholar
Kita, Sotaro
(2003) Pointing: Where language, culture, and cognition meet. Mahwah, NJ: Lawrence Erlbaum. DOI logoGoogle Scholar
Kita, Sotaro & Hedda Lausberg
(2008) Generation of co-speech gestures based on spatial imagery from the right-hemisphere: Evidence from split-brain patients. Cortex, 441, 131–39. DOI logoGoogle Scholar
Koffka, Kurt
(1936) Principles of Gestalt psychology. New York: Harcourt Brace.Google Scholar
Kranstedt, Alfred, Andy Lücking, Thies Pfeiffer, Hannes Rieser, & Marc Staudacher
(2006) Measuring and reconstructing pointing in visual contexts. In Proceedings of the Brandial 2006 – The 10th Workshop on the Semantics and Pragmatics of Dialogue (pp. 82–89). Potsdam: Universitätsverlag.Google Scholar
Lausberg, Hedda & Han Sloetjes
(2009) Coding gestural behavior with the NEUROGES-ELAN system. Behavior Research Methods, 41 (3), 841–849. DOI logoGoogle Scholar
Le Guen, Olivier
(2011) Modes of pointing to existing spaces and the use of frames of reference. Gesture, 11 (3), 271–307. DOI logoGoogle Scholar
Liddell, Scott K.
(2003) Grammar, gesture, and meaning in American Sign Language. Cambridge: Cambridge University Press. DOI logoGoogle Scholar
Lücking, Andy, Kirsten Bergmann, Florian Hahn, Stefan Kopp, & Hannes Rieser
(2010) The Bielefeld Speech and Gesture Alignment Corpus (SaGA). In 4th Workshop on the Representation and Processing of Sign Languages (LREC 2010): Corpora and Sign Language Technologies. University of Hamburg.Google Scholar
Mandel, Mark A.
(1977) Iconic devices in American Sign Language. In Lynn A. Friedman (Ed.), On the other hand: New perspectives on American Sign Language (pp. 57–107). New York: Academic Press.Google Scholar
McNeill, David
(1992) Hand and mind: What gestures reveal about thought. Chicago: The University of Chicago Press.Google Scholar
Mittelberg, Irene & Linda R. Waugh
(2009) Metonymy first, metaphor second: A cognitive-semiotic approach to multimodal figures of thought in co-speech gesture. In Charles Forceville & Eduardo Urios-Aparisi (Eds.), Multimodal metaphor (pp. 329–356). Berlin & New York: Mouton de Gruyter.Google Scholar
Müller, Cornelia
(1998) Redebegleitende Gesten. Kulturgeschichte – Theorie – Sprachvergleich. Berlin: Arno Spitz.Google Scholar
(2008) What gestures reveal about the nature of metaphor. In Alan Cienki & Cornelia Müller (Eds.), Metaphor and gesture (pp. 219–245). Amsterdam & Philadelphia: John Benjamins. DOI logoGoogle Scholar
(2014) Gestural modes of representation as techniques of depiction. In Cornelia Müller, Alan Cienki, Ellen Fricke, Silva H. Ladewig, David McNeill, & Jana Bressem (Eds.), Body – language – communication: An international handbook on multimodality in human interaction (Vol. 21, pp. 1687–1702). Berlin & Boston: Mouton de Gruyter.Google Scholar
Pfeiffer, Thies
(2010) Understanding multimodal deixis with gaze and gesture in conversational interfaces. Doctoral Dissertation, Bielefeld University.Google Scholar
Salffner, Sophie
(2014) Farming, food and yam: language and cultural practices among Ikaan speakers: An archive of language and cultural material from the Akaan people of Ikakumo (Ondo State and Edo State, Nigeria). London: SOAS, Endangered Languages Archive. [URL]
Streeck, Jürgen
(2009) Gesturecraft: The manu-facture of meaning. Amsterdam: John Benjamins. DOI logoGoogle Scholar
Sowa, Timo
(2006) Understanding coverbal iconic gestures in shape descriptions. Doctoral Dissertation, University of Bielefeld.Google Scholar
Talmy, Leonard
(2000) Toward a cognitive semantics, Vol. 1: Concept structuring systems. Cambridge: MIT Press.Google Scholar
(2018) The targeting system of language. Cambridge: MIT Press. DOI logoGoogle Scholar
Wagemans, Johan, James H. Elder, Michael Kubovy, Stephen E. Palmer, Mary A. Peterson, Manish Singh, & Joachim von der Heydt
(2012) A century of Gestalt psychology in visual perception I. Perceptual grouping and figure-ground organization. Psychological Bulletin, 138 (6), 1172–1217. DOI logoGoogle Scholar
Wilkins, David P.
(2003) Why pointing with the index finger is not a universal (in sociocultural and semiotic terms). In Sotaro Kita (Ed.), Pointing: Where language, culture, and cognition meet (pp. 171–216). Mahwah, NJ: Lawrence Erlbaum.Google Scholar
Cited by

Cited by 4 other publications

Abdel-Raheem, Ahmed
2023. Do political cartoons and illustrations have their own specialized forms for warnings, threats, and the like? Speech acts in the nonverbal mode. Social Semiotics 33:1 ► pp. 64 ff. DOI logo
Hassemer, Julius & Vito Evola
2021. Cognitive Linguistics and Gesture. In The Routledge Handbook of Cognitive Linguistics, ► pp. 512 ff. DOI logo
Mesh, Kate, Emiliana Cruz, Joost van de Weijer, Niclas Burenhult & Marianne Gullberg
2021. Effects of Scale on Multimodal Deixis: Evidence From Quiahije Chatino. Frontiers in Psychology 11 DOI logo
Winter, Bodo & Tyler Marghetis
2023. Multimodality matters in numerical communication. Frontiers in Psychology 14 DOI logo

This list is based on CrossRef data as of 30 march 2024. Please note that it may not be complete. Sources presented here have been supplied by the respective publishers. Any errors therein should be reported to them.