Article published In:
GestureVol. 8:3 (2008) ► pp.285–301
Depicting by gesture
This paper deals with ways in which gestural “pictures” are made, i.e., manual depictions of phenomena in the world. The view that “iconic” gestures uniformly function by way of some resemblance between signifier and signified is rejected, giving way to an understanding of depiction by gesture as the achievement of a heterogeneous set of practices, some of which rely on relations of contiguity or indexicality to evoke commonly known objects or scenes. Others seem to be derivative of other representation methods (e.g., drawing on surfaces). The paper reviews some existing work on gestural depiction methods, offers a working heuristics, and illustrates some of its categories. It is suggested that some of the basic ways in which actions of the hands evoke the world in gesture correspond to fundamental modes of existence and activity of human hands in the world: hands depict by enacting their familiar, “real-world” capacities as users, transporters, experiencers, assemblers, molders, and shapers of things.
Keywords: description, iconicity, depiction, gesture, indexicality
Cited by
Cited by 104 other publications
Bazzaro, Florence, Sebastien Chevriau & Marjorie Charrier
2020. 2020 22nd Symposium on Virtual and Augmented Reality (SVR), ► pp. 469 ff. 
Bergmann, Kirsten & Stefan Kopp
2009. GNetIc – Using Bayesian Decision Networks for Iconic Gesture Generation. In Intelligent Virtual Agents [Lecture Notes in Computer Science, 5773], ► pp. 76 ff.
Bergmann, Kirsten & Stefan Kopp
2010. MODELING THE PRODUCTION OF COVERBAL ICONIC GESTURES BY LEARNING BAYESIAN DECISION NETWORKS. Applied Artificial Intelligence 24:6 ► pp. 530 ff.
Bergmann, Kirsten & Stefan Kopp
2010. Systematicity and Idiosyncrasy in Iconic Gesture Use: Empirical Analysis and Computational Modeling. In Gesture in Embodied Communication and Human-Computer Interaction [Lecture Notes in Computer Science, 5934], ► pp. 182 ff.
Bergmann, Kirsten, Stefan Kopp & Friederike Eyssel
2010. Individualized Gesturing Outperforms Average Gesturing – Evaluating Gesture Production in Virtual Humans. In Intelligent Virtual Agents [Lecture Notes in Computer Science, 6356], ► pp. 104 ff.
Bergmann, Kirsten, Maria del Mar Cordero Rull & Birgit Lugrin
2018. Proceedings of the 4th International Workshop on Multimodal Analyses Enabling Artificial Agents in Human-Machine Interaction, ► pp. 26 ff.
Bezemer, Jeff, Ged Murtagh & Alexandra Cope
2019. Inspecting Objects: Visibility Manoeuvres in Laparoscopic Surgery. In Embodied Activities in Face-to-face and Mediated Settings, ► pp. 107 ff.
Bressem, Jana, Silva H. Ladewig & Cornelia Müller
2018. Ways of expressing action in multimodal narrations – the semiotic complexity of character viewpoint depictions. In Linguistic Foundations of Narration in Spoken and Sign Languages [Linguistik Aktuell/Linguistics Today, 247], ► pp. 223 ff.
Calpin, Nicole & Jessica Menold
2023. The Cognitive Costs of Design Tasks: Examining Cognitive Load Through Verbal and Physical Indicators. Journal of Mechanical Design 145:10
Cash, Philip & Anja Maier
2016. Prototyping with your hands: the many roles of gesture in the communication of design concepts. Journal of Engineering Design 27:1-3 ► pp. 118 ff.
Chiou, Erin K., Mustafa Demir, Verica Buchanan, Christopher C. Corral, Mica R. Endsley, Glenn J. Lematta, Nancy J. Cooke & Nathan J. McNeese
2022. Towards Human–Robot Teaming: Tradeoffs of Explanation-Based Communication Strategies in a Virtual Search and Rescue Task. International Journal of Social Robotics 14:5 ► pp. 1117 ff.
Chiou, Erin K., Eric Holder, Igor Dolgov, Kaleb McDowell, Lance Menthe, Rod D. Roscoe & Shivam Zaveri
2020. Human, AI, Robot Teaming and the Future of Work: Barriers and Opportunities for Advancement. Proceedings of the Human Factors and Ergonomics Society Annual Meeting 64:1 ► pp. 62 ff.
Chiou, Erin K. & John D. Lee
2016. Cooperation in Human-Agent Systems to Support Resilience. Human Factors: The Journal of the Human Factors and Ergonomics Society 58:6 ► pp. 846 ff.
Choksi, Nishaant
2020. Expressives and the multimodal depiction of social types in Mundari. Language in Society 49:3 ► pp. 379 ff.
Cibulka, Paul
Cienki, Alan
2022. The study of gesture in cognitive linguistics: How it could inform and inspire other research in cognitive science. WIREs Cognitive Science 13:6
Coughlan, Johannes
2023. Design and the Polysemy of Value: On a problem within the language of valuation studies. Valuation Studies 10:1 ► pp. 167 ff.
Cuffari, Elena
2012. Gestural sense-making: hand gestures as intersubjective linguistic enactments. Phenomenology and the Cognitive Sciences 11:4 ► pp. 599 ff.
DANİŞMAN, Şahin & Dilek TANIŞLI
2023. Semiotic Examination of Algebra Teaching Process in the Context of Gestures. Anadolu Üniversitesi Eğitim Fakültesi Dergisi 7:4 ► pp. 1065 ff.
del Puy Carretero, María, Aitor Ardanza, Sara García, Helen Díez, David Oyarzun & Nuria Ruiz
2012. Improving Gestural Communication in Virtual Characters. In Articulated Motion and Deformable Objects [Lecture Notes in Computer Science, 7378], ► pp. 69 ff.
Dingemanse, Mark
Evans, Bryn
2017. Chapter 11. Intercorporeal (re)enaction. In Moving Bodies in Interaction – Interacting Bodies in Motion [Advances in Interaction Studies, 8], ► pp. 267 ff.
Gibson, Will
2020. Sensory communication in YouTube reviews: The interactional construction of products. Discourse & Communication 14:4 ► pp. 383 ff.
Giorgolo, Gianluca
2012. Integration of Gesture and Verbal Language: A Formal Semantics Approach. In Gesture and Sign Language in Human-Computer Interaction and Embodied Communication [Lecture Notes in Computer Science, 7206], ► pp. 216 ff.
Goldstein, Donna M., Kira Hall & Matthew Bruce Ingram
2020. Trump’s Comedic Gestures as Political Weapon. In Language in the Trump Era, ► pp. 97 ff.
Gries, Stefan Th. & Andrea L. Berez
2017. Linguistic Annotation in/for Corpus Linguistics. In Handbook of Linguistic Annotation, ► pp. 379 ff.
Hall, Kira, Donna M. Goldstein & Matthew Bruce Ingram
2016. The hands of Donald Trump. HAU: Journal of Ethnographic Theory 6:2 ► pp. 71 ff.
Harrison, Simon
2021. Showing as sense-making in oral presentations: The speech-gesture-slide interplay in TED talks by Professor Brian Cox. Journal of English for Academic Purposes 53 ► pp. 101002 ff.
Harrison, Simon
2023. Sensing gesture’s relationality. Review of Jürgen Streeck, Self-making Man: A Day of Action, Life and Language, Cambridge: Cambridge University Press, 2017. Phenomenology and the Cognitive Sciences
Hassemer, Julius & Bodo Winter
Hassemer, Julius & Bodo Winter
2018. Decoding Gestural Iconicity . Cognitive Science 42:8 ► pp. 3034 ff.
Heller, Vivien & Katharina J. Rohlfing
2017. Reference as an Interactive Achievement: Sequential and Longitudinal Analyses of Labeling Interactions in Shared Book Reading and Free Play. Frontiers in Psychology 8
Hodge, Gabrielle & Lindsay Ferrara
2022. Iconicity as Multimodal, Polysemiotic, and Plurifunctional. Frontiers in Psychology 13
Hodge, Gabrielle & Trevor Johnston
2014. Points, Depictions, Gestures and Enactment: Partly Lexical and Non-Lexical Signs as Core Elements of Single Clause-Like Units in Auslan (Australian Sign Language). Australian Journal of Linguistics 34:2 ► pp. 262 ff.
Hsu, Hui-Chieh, Geert Brône & Kurt Feyaerts
2021.
In other gestures: Multimodal iteration in cello master classes. Linguistics Vanguard 7:s4
Hsu, Hui-Chieh, Geert Brône & Kurt Feyaerts
2021. When Gesture “Takes Over”: Speech-Embedded Nonverbal Depictions in Multimodal Interaction. Frontiers in Psychology 11
Ingram, Matthew Bruce & Ian Mitchell Wallace
2022. Making a mark: Transforming everyday work of city arborists into dance. Frontiers in Communication 7
Katila, Julia & Johanne S. Philipsen
Keevallik, Leelo, Emily Hofstetter, Agnes Löfgren & Sally Wiggins
2024. Repetition for real-time coordination of action: Lexical and non-lexical vocalizations in collaborative time management. Discourse Studies
Kendon, Adam
2023. Pina Boggi Cavallo and My “Southern Italian Gesture Project”. In Humanity in Psychology [Theory and History in the Human and Social Sciences, ], ► pp. 189 ff.
Kinalzik, Noelle & Vivien Heller
2020. Establishing joint imagined spaces in game explanations. Research on Children and Social Interaction 4:1 ► pp. 28 ff.
Kiose, Maria, Anna Leonteva & Olga Agafonova
2022. Aesthetic multimodality of speech and gesture: Towards its functional framework. Languages and Modalities 2 ► pp. 1 ff.
Kopp, Stefan & Kirsten Bergmann
2012. Individualized Gesture Production in Embodied Conversational Agents. In Human-Computer Interaction: The Agency Perspective [Studies in Computational Intelligence, 396], ► pp. 287 ff.
Korkiakangas, Terhi K. & John P. Rae
2013. Gearing up to a New Activity: How Teachers use Object Adjustments
to Manage the Attention of Children with Autism. Augmentative and Alternative Communication 29:1 ► pp. 83 ff.
Kosmala, Loulou
2020. (Dis)fluencies and their contribution to the co-construction of meaning in native and non-native tandem interactions of French and English. TIPA. Travaux interdisciplinaires sur la parole et le langage :36
Kosmala, Loulou, F. Neveu, B. Harmegnies, L. Hriba, S. Prévost & A. Steuckardt
2020. Euh le saviez-vous ? le rôle des (dis)fluences en contexte interactionnel : étude exploratoire et qualitative. SHS Web of Conferences 78 ► pp. 01018 ff.
KUMCU, Alper
2020. Konferans Salonunda Dağıtık Biliş: Andaş Çeviri Sürecine Yeni Bir Bakış. Hacettepe Üniversitesi Edebiyat Fakültesi Dergisi 37:1 ► pp. 170 ff.
Kurz, Kim B., Geo Kartheiser & Peter C. Hauser
2023. Second language learning of depiction in a different modality: The case of sign language acquisition. Frontiers in Communication 7
Leonteva, Anna & Tatiana Sokoreva
2022. Nonverbal Constituents of Argumentative Discourse: Gesture and Prosody Interaction. In Speech and Computer [Lecture Notes in Computer Science, 13721], ► pp. 416 ff.
Lopez-Ozieblo, Renia
2018. Disagreeing without a ‘no’: How teachers indicate disagreement in a Hong Kong classroom. Journal of Pragmatics 137 ► pp. 1 ff.
Lopez-Ozieblo, Renia
2018. Can gestures help clarify the meaning of the Spanish marker ‘se’?. Lingua 208 ► pp. 1 ff.
Lu, Jenny C. & Susan Goldin-Meadow
2018. Creating Images With the Stroke of a Hand: Depiction of Size and Shape in Sign Language. Frontiers in Psychology 9
Luck, Rachael
2014. Seeing architecture in action: designing, evoking, and depicting space and form in embodied interaction. International Journal of Design Creativity and Innovation 2:3 ► pp. 165 ff.
Löfgren, Agnes
2023. Relocating to Depict: Managing the Interactional Agenda at Opera Rehearsals. Research on Language and Social Interaction 56:3 ► pp. 209 ff.
Löfgren, Agnes & Emily Hofstetter
2023. Introversive semiosis in action: depictions in opera rehearsals. Social Semiotics 33:3 ► pp. 601 ff.
Lücking, Andy, Kirsten Bergman, Florian Hahn, Stefan Kopp & Hannes Rieser
2013. Data-based analysis of speech and gesture: the Bielefeld Speech and Gesture Alignment corpus (SaGA) and its applications. Journal on Multimodal User Interfaces 7:1-2 ► pp. 5 ff.
Marentette, Paula, Paola Pettenati, Arianna Bello & Virginia Volterra
2016. Gesture and Symbolic Representation in Italian and English‐Speaking Canadian 2‐Year‐Olds. Child Development 87:3 ► pp. 944 ff.
Masson-Carro, Ingrid, Martijn Goudbeek & Emiel Krahmer
2016. Can you handle this? The impact of object affordances on how co-speech gestures are produced. Language, Cognition and Neuroscience 31:3 ► pp. 430 ff.
Masson-Carro, Ingrid, Martijn Goudbeek & Emiel Krahmer
2017. How What We See and What We Know Influence Iconic Gesture Production. Journal of Nonverbal Behavior 41:4 ► pp. 367 ff.
Janet McIntosh & Norma Mendoza-Denton
2020. Language in the Trump Era,
Mehler, Alexander, Andy Lücking & Giuseppe Abrami
2015. WikiNect: image schemata as a basis of gestural writing for kinetic museum wikis. Universal Access in the Information Society 14:3 ► pp. 333 ff.
Morgenstern, Aliyah
2019. Chapitre 2. Le développement multimodal du langage de l’enfant : des premiers bourgeons aux constructions multimodales. In Multimodalité du langage dans les interactions et l’acquisition,
Napoli, Donna Jo & Casey Ferrara
2021. Correlations Between Handshape and Movement in Sign Languages. Cognitive Science 45:5
Napoli, Donna Jo & Lorraine Leeson
2020. Visuo-spatial construals that aid in understanding activity in visual-centred narrative. Language, Cognition and Neuroscience 35:4 ► pp. 440 ff.
Nicoladis, Elena, Ashmeen Aneja, Jasmine Sidhu & Avnit Dhanoa
2022. Is There a Correlation Between the Use of Representational Gestures and Self-adaptors?. Journal of Nonverbal Behavior 46:3 ► pp. 269 ff.
Nicoladis, Elena & Emma Hill
NIU, Chenxi, Alan CIENKI, Gerardo ORTEGA & Martine COENE
2024. Learning to express causal events in Mandarin Chinese: A multimodal perspective. Journal of Child Language 51:1 ► pp. 191 ff.
Nuckolls, Janis B.
2019. Chapter 7. The sensori-semantic clustering of ideophonic meaning in Pastaza Quichua. In Ideophones, Mimetics and Expressives [Iconicity in Language and Literature, 16], ► pp. 167 ff.
Nuckolls, Janis B.
Nyst, Victoria
2016. The depiction of size and shape in gestures accompanying object descriptions in Anyi (Côte d’Ivoire) and in Dutch (The Netherlands). Gesture 15:2 ► pp. 156 ff.
Nyst, Victoria
2016. Size and shape depictions in the manual modality: A taxonomy of iconic devices in Adamorobe Sign Language. Semiotica 2016:210
Oben, Bert
2018. Chapter 10. Gaze as a predictor for lexical and gestural alignment. In Eye-tracking in Interaction [Advances in Interaction Studies, 10], ► pp. 233 ff.
OBEN, BERT & GEERT BRÔNE
2015. What you see is what you do: on the relationship between gaze and gesture in multimodal alignment. Language and Cognition 7:4 ► pp. 546 ff.
Oben, Bert & Geert Brône
2016. Explaining interactive alignment: A multimodal and multifactorial account. Journal of Pragmatics 104 ► pp. 32 ff.
Poulin, M., C. Masclet & J.-F. Boujut
2020. THE CO-DESIGN COGNITIVE PROCESS: IMPACTS OF A SPATIAL AUGMENTED REALITY PLATFORM. Proceedings of the Design Society: DESIGN Conference 1 ► pp. 1047 ff.
Poulin, Maud, Cédric Masclet & Jean-François Boujut
2024. Investigating the effects of spatial augmented reality on user participation in co-design sessions: A case study. Computers in Industry 154 ► pp. 104023 ff.
Pouw, Wim, Lisette de Jonge‐Hoekstra, Steven J. Harrison, Alexandra Paxton & James A. Dixon
2021. Gesture–speech physics in fluent speech and rhythmic upper limb movements. Annals of the New York Academy of Sciences 1491:1 ► pp. 89 ff.
Pouw, Wim, Jan de Wit, Sara Bögels, Marlou Rasenberg, Branka Milivojevic & Asli Ozyurek
2021. Semantically Related Gestures Move Alike: Towards a Distributional Semantics of Gesture Kinematics. In Digital Human Modeling and Applications in Health, Safety, Ergonomics and Risk Management. Human Body, Motion and Behavior [Lecture Notes in Computer Science, 12777], ► pp. 269 ff.
Quasinowski, Benjamin, Solmaz Assa, Cadja Bachmann, Wei Chen, Melih Elcin, Caner Kamisli, Tao Liu, Alexander H. Maass, Stefanie Merse, Caroline Morbach, Anja Neumann, Till Neumann, Ilka Sommer, Stefan Stoerk, Sarah Weingartz, Anja Weiss & Goetz Wietasch
2023. Hearts in their hands—Physicians’ gestures embodying shared professional knowledge around the world. Sociology of Health & Illness 45:5 ► pp. 1101 ff.
Quezada, Percy, Rony Cueva & Freddy Paz
2021. A Systematic Review of User-Centered Design Techniques Applied to the Design of Mobile Application User Interfaces. In Design, User Experience, and Usability: UX Research and Design [Lecture Notes in Computer Science, 12779], ► pp. 100 ff.
Rasenberg, Marlou, Asli Özyürek & Mark Dingemanse
2020. Alignment in Multimodal Interaction: An Integrative Framework. Cognitive Science 44:11
Katharina J. Rohlfing
2023. Eine längsschnittliche Betrachtung des integrierten Gesten-Lautsprach-Systems in verschiedenen Diskurspraktiken. Sprache · Stimme · Gehör 47:04 ► pp. 199 ff.
Schmidt, Axel & Arnulf Deppermann
2023. Showing and telling—How directors combine embodied demonstrations and verbal descriptions to instruct in theater rehearsals. Frontiers in Communication 7
Sekine, Kazuki, Catharine Wood & Sotaro Kita
2018. Gestural depiction of motion events in narrative increases symbolic distance with age. Language, Interaction and Acquisition 9:1 ► pp. 40 ff.
Shaw, Emily
Steier, Rolf
2014. Posing the Question: Visitor Posing as Embodied Interpretation in an Art Museum. Mind, Culture, and Activity 21:2 ► pp. 148 ff.
Stevens, Michael Paul & Simon Harrison
Streeck, Jürgen
2009. Forward-Gesturing. Discourse Processes 46:2-3 ► pp. 161 ff.
Streeck, Jürgen
2013. Interaction and the living body. Journal of Pragmatics 46:1 ► pp. 69 ff.
Streeck, Jürgen
2015. Embodiment in Human Communication. Annual Review of Anthropology 44:1 ► pp. 419 ff.
Streeck, Jürgen
Stromberg, Peter G.
2021. Enacting Culture. Current Anthropology 62:4 ► pp. 426 ff.
Tampubolon, Manotar
2021. Human Interaction Model Emerging Changes During the COVID-19 Pandemic: A Qualitative Study. SSRN Electronic Journal
Tutt, Dylan & Jon Hindmarsh
2011. Reenactments at Work: Demonstrating Conduct in Data Sessions. Research on Language & Social Interaction 44:3 ► pp. 211 ff.
Urbanik, Paweł & Jan Svennevig
2021. Action-Depicting Gestures and Morphosyntax: The Function of Gesture-Speech Alignment in the Conversational Turn. Frontiers in Psychology 12
van Nispen, Karin, W.M.E. Mieke, E. van de Sandt‐Koenderman & Emiel Krahmer
2018. The comprehensibility of pantomimes produced by people with aphasia. International Journal of Language & Communication Disorders 53:1 ► pp. 85 ff.
Vazquez, Alicia Nahmad
2022. Evaluating Team Fluency in Human-Industrial Robot Collaborative Design Tasks. In Computer-Aided Architectural Design. Design Imperatives: The Future is Now [Communications in Computer and Information Science, 1465], ► pp. 378 ff.
Wagner, Petra, Zofia Malisz & Stefan Kopp
2014. Gesture and speech in interaction: An overview. Speech Communication 57 ► pp. 209 ff.
Weidinger, Nicole, Katrin Lindner, Katharina Hogrefe, Wolfram Ziegler & Georg Goldenberg
2017. Getting a Grasp on Children’s Representational Capacities in Pantomime of Object Use. Journal of Cognition and Development 18:2 ► pp. 246 ff.
Wilkinson, Ray
2013. Gestural Depiction in Acquired Language Disorders: On the Form
and Use of Iconic Gestures in Aphasic Talk-In-Interaction. Augmentative and Alternative Communication 29:1 ► pp. 68 ff.
Wu, Suwei
2019. The Tool Noun and Tool Verb Alternation in Gesture. Multimodal Communication 8:2
Wu, Ying Choon & Seana Coulson
2015. Iconic Gestures Facilitate Discourse Comprehension in Individuals With Superior Immediate Memory for Body Configurations. Psychological Science 26:11 ► pp. 1717 ff.
[no author supplied]
[no author supplied]
This list is based on CrossRef data as of 30 march 2024. Please note that it may not be complete. Sources presented here have been supplied by the respective publishers. Any errors therein should be reported to them.