Article published In:
GestureVol. 14:3 (2014) ► pp.351–374
The haptic nature of gesture
Rethinking gesture with new multitouch digital technologies
This paper is about re-thinking gesture in order to reckon with its material and haptic nature, especially in the current multitouch technology environment. This re-thinking of gesture returns to the principle of indexicality found in Peirce’s material semiotics, and develops this principle through the work of Gilles Châtelet and Gilles Deleuze around hand-eye relationships. Drawing on the work of Jürgen Streek, we propose and discuss the notion of the tangible gesture, in the context of mathematical explorations of young children with a multitouch iPad environment designed to promote counting on and with the fingers.
Keywords: technology, mathematics, the senses
References (33)
Alibali, Martha & Allysa DiRusso (1999). The function of gesture in learning to count: More than keeping track. Cognitive Development, 141, 37–56. 
Angel, Maria & Anna Gibbs (2013). At the time of writing: Digital media, gestures, and handwriting. Electronic Book Review. Retrieved 22.09.2013. [URL]
Bartolini-Bussi, Mariolina & Mara Boni (2003). Instruments for semiotic mediation in primary school classrooms. For the Learning of Mathematics, 23 (2), 12–19.
Châtelet, Gilles ([1993] 2000). Les enjeux du mobile. Paris: Seuil. English translation by Robert Shore & Muriel Zagha (2000): Figuring space: Philosophy, mathematics, and physics. Dordrecht: Kluwer, 2000.
(2010). L’enchantement du virtuel. Philosophie, physique, mathématique. Paris, France: Presses de l’École normale supérieure.
de Freitas, Elizabeth & Nathalie Sinclair (2014). Mathematics and the body: Material entanglements in the classroom. New York: Cambridge University Press.
Deleuze, Gilles. (2003). Francis Bacon: The logic of sensation. Translated from the French by Daniel W. Smith. London, UK: Continuum.
Gibson, James J. (1966). The senses considered as perceptual systems. Boston: Houghton Mifflin.
Goodwin, Charles (2007). Environmentally coupled gestures. In Susan D. Duncan, Justine Cassell, & Elena T. Levy (Eds.), Gesture and the dynamic dimension of language: Essays in honor of David McNeill (pp. 195–212). Amsterdam: John Benjamins.
Kendon, Adam (2000). Language and gesture: Unity or duality? In David McNeill (Ed.), Language and gesture (pp. 47–63). Cambridge, UK: Cambridge University Press.
(2004). Gesture: Visible action as utterance. Cambridge, UK: Cambridge University Press.
Kita, Sotaro (Ed.) (2003). Pointing: Where language, culture, and cognition meet. Mahwah, NJ: Erlbaum.
Maturana, Humberto & Francisco Varela ([1984] 1992). The tree of knowledge: Biological roots of human understanding. Boston, MA: Shambhala Publications.
McNeill, David (1992). Hand and mind: What gestures reveal about thought. Chicago: University of Chicago Press.
(2005). Gesture and thought. Chicago: University of Chicago Press.
Mulder, Axel (1996). Hand gestures for HCI. Technical Report, NSERC Hand Centered Studies of Human Movement project. Burnaby, BC, Canada: Simon Fraser University. Available: [URL]
Nemirovsky, Ricardo, Molly Kelton, & Bohdan Rhodehamel (2012). Gesture and imagination: On the constitution and uses of phantasms. Gesture, 12 (2), 130–165.
Noel, Marie-Pascale (2005). Finger gnosia: A predictor of numerical abilities in children? Child Neuropsychology, 51, 413–430.
Peirce, Charles Sanders ([1894] 1998). What is a sign? In Nathan Houser, et al. (Eds.), The essential Peirce, Vol. 21 (pp. 4–10). Bloomington, IN: Indiana University Press.
(1932).
Elements of logic
. In Charles Hartshorne & Paul Weiss (Eds.), Collected papers of Charles Sanders Peirce, Vol. 21. Cambridge: Harvard University Press.
(1994). Peirce on signs: Writings on semiotic, James Hoopes (Ed.). Chapel Hill, NC: University of North Carolina Press.
Radford, Luis (2002). The seen, the spoken and the written: A semiotic approach to the problem of objectification of mathematical knowledge. For the Learning of Mathematics, 22 (2), 14–23.
Rasmussen, Chris, Michelle Stephan, & Karen Allen (2004). Classroom mathematical practices and gesturing. Journal of Mathematical Behavior, 231, 301–323.
Reynolds, Fione J. & Robert A. Reeve (2002). Gesture in collaborative mathematical problem-solving. Journal of Mathematical Behavior, 201, 447–460.
Robutti, Ornella & Francesca Ferrara (2002). Approaching algebra through motion experiences. In Anne D. Cockburn & Elena Nardi (Eds.),
Proceedings of the 26th PME International Conference
, Vol. 11 (pp. 111–118).
Rossini, Nicla (2012). Reinterpreting gesture as language: Language in action. Amsterdam: IOS Press.
Roth, Wolff-Michael (2001). Gestures: Their role in teaching and learning. Review of Educational Research, 71 (3), 365–392.
Rotman, Brian (1987). Signifying nothing: A semiotics of zero. Basingstoke: Macmillan.
(2008). Becoming beside ourselves: The alphabet, ghosts, and distributed human beings. Durham: Duke University Press.
Sinclair, Nathalie & Nicholas Jackiw (2011). Touchcounts [Software application for the iPad]. Burnaby, BC: Tangible Learning Project.
Sinclair, Nathalie & David Pimm (2014). Number’s subtle touch: Expanding finger gnosis in the era of multi-touch technologies. In Nicol Cynthia et al. (Eds.),
Proceedings of the Psychology of Mathematics Education Conference
. Vancouver, Canada, July 15–20, 2014, Vol. 51 (pp. 209–216).
Streeck, Jürgen (2009). Gesturecraft: The manu-facture of meaning. Amsterdam: John Benjamins.
Villamor, Craig, Dan Willis, & Luke Wroblewski (2010). Touch gesture reference guide. [URL]
Cited by (44)
Cited by 44 other publications
Gerner, Alexander Matthias, Thiemo Breyer, Niklas Grouls & Johannes F. M. Schick
2024. Diachronic Perspectives on Embodiment and Technology: Gestures and Artefacts – An Introduction. In Diachronic Perspectives on Embodiment and Technology [Philosophy of Engineering and Technology, 46], ► pp. 1 ff. 
Kynigos, Chronis
2024. Embedding Mathematics in Socio-Scientific Games: The Mathematical in Grappling with Wicked Problems. Education Sciences 14:6 ► pp. 630 ff.
Miller, Marty
2024. Reflecting the seen scene: Theorizing screenshots as multistable metapictures. Convergence: The International Journal of Research into New Media Technologies
Walkington, Candace, Mitchell J. Nathan, Jonathan Hunnicutt, Julianna Washington & Monique Zhou
2024. New kinds of embodied interactions that arise in augmented reality dynamic geometry software. The Journal of Mathematical Behavior 75 ► pp. 101175 ff.
Abrahamson, Dor, Kimiko Ryokai & Justin Dimmel
2023. Learning Mathematics with Digital Resources: Reclaiming the Cognitive Role of Physical Movement. In Handbook of Digital Resources in Mathematics Education [Springer International Handbooks of Education, ], ► pp. 1 ff.
Abrahamson, Dor, Kimiko Ryokai & Justin Dimmel
2024. Learning Mathematics with Digital Resources: Reclaiming the Cognitive Role of Physical Movement. In Handbook of Digital Resources in Mathematics Education [Springer International Handbooks of Education, ], ► pp. 609 ff.
de Freitas, Elizabeth & Nathalie Sinclair
2023. Landing Sites, Cities, and Nonplaces: Collaborating Across the Conference Circuit. Qualitative Inquiry 29:1 ► pp. 7 ff.
Sinha, Ravi, Hillary Swanson, Jody Clarke-Midura, Jessica F. Shumway, Victor R. Lee & Sanjay Chandrasekharan
2023. Proceedings of the 2023 Symposium on Learning, Design and Technology, ► pp. 1 ff.
Yeung, Wing-Leung & Oi-Lam Ng
2023. Characterizing touchscreen actions in technology-enhanced embodied learning for mathematics instruction in K-12 setting – A systematic review (2010–2023). Computers & Education 205 ► pp. 104881 ff.
Robutti, Ornella, Cristina Sabena, Christina Krause, Carlotta Soldano & Ferdinando Arzarello
2022. Gestures in Mathematics Thinking and Learning. In Handbook of Cognitive Mathematics, ► pp. 685 ff.
Robutti, Ornella, Cristina Sabena, Christina Krause, Carlotta Soldano & Ferdinando Arzarello
2022. Gestures in Mathematics Thinking and Learning. In Handbook of Cognitive Mathematics, ► pp. 1 ff.
Sinclair, Nathalie, Mariam Haspekian, Ornella Robutti & Alison Clark-Wilson
2022. Revisiting Theories That Frame Research on Teaching Mathematics with Digital Technology. In The Mathematics Teacher in the Digital Era [Mathematics Education in the Digital Era, 16], ► pp. 391 ff.
Sinclair, Nathalie & Eva Jablonka
2022. Mathematics Learning: Structured Ways of Moving With. In The Palgrave Handbook of Embodiment and Learning, ► pp. 419 ff.
Tucker, Stephen I. & Teri N. Johnson
2022. Developing number sense with Fingu: a preschooler’s embodied mathematics during interactions with a multi-touch digital game. Mathematics Education Research Journal 34:2 ► pp. 393 ff.
Bairral, Marcelo
2021. Interacciones, toques en pantalla y aprendizaje de cuadriláteros. Revista Venezolana de Investigación en Educación Matemática 1:2 ► pp. 1 ff.
Billion, Lara Kristina
2021. Reconstruction of the Interpretation of Geometric Diagrams of Primary School Children Based on Actions on Various Materials -<br /> A Semiotic Perspective on Actions. International Electronic Journal of Mathematics Education 16:3 ► pp. em0650 ff.
Forte, Magali
2021. Identités sociomatérielles et création d’histoires plurilingues et numériques. The Canadian Modern Language Review 77:4 ► pp. 332 ff.
Güneş, Canan
2021. A Quantitative Shift Towards Multiplicative Thinking. Digital Experiences in Mathematics Education 7:3 ► pp. 361 ff.
Karnam, DurgaPrasad, Harshit Agrawal, Pranay Parte, Saurabh Ranjan, Priyanka Borar, Prasanna Prakash Kurup, Amose Jebin Joel, Pattamadai Sankaran Srinivasan, Uddhav Suryawanshi, Aniket Sule & Sanjay Chandrasekharan
2021. Touchy feely vectors: A compensatory design approach to support model‐based reasoning in developing country classrooms. Journal of Computer Assisted Learning 37:2 ► pp. 446 ff.
Pande, Prajakt
2021. Learning and expertise with scientific external representations: an embodied and extended cognition model. Phenomenology and the Cognitive Sciences 20:3 ► pp. 463 ff.
Shvarts, Anna, Rosa Alberto, Arthur Bakker, Michiel Doorman & Paul Drijvers
2021. Embodied instrumentation in learning mathematics as the genesis of a body-artifact functional system. Educational Studies in Mathematics 107:3 ► pp. 447 ff.
Tancredi, Sofia, Rotem Abdu, Dor Abrahamson & Ramesh Balasubramaniam
2021. Modeling nonlinear dynamics of fluency development in an embodied-design mathematics learning environment with Recurrence Quantification Analysis. International Journal of Child-Computer Interaction 29 ► pp. 100297 ff.
Tucker, Stephen I. & Patricia S. Moyer-Packenham
2021. Exploring the phenomenon of distance in children’s interactions with touchscreen digital mathematics games. International Journal of Mathematical Education in Science and Technology 52:10 ► pp. 1447 ff.
Bakos, Sandy & David Pimm
2020. Beginning to Multiply (with) Dynamic Digits: Fingers as Physical–Digital Hybrids. Digital Experiences in Mathematics Education 6:2 ► pp. 145 ff.
Janßen, Thomas, Estela Vallejo-Vargas, Angelika Bikner-Ahsbahs & David A. Reid
2020. Design and Investigation of a Touch Gesture for Dividing in a Virtual Manipulative Model for Equation-solving. Digital Experiences in Mathematics Education 6:2 ► pp. 166 ff.
Kelleher, William
2020. Digital media, the body and agency in a South African education institution from the perspective of narrative research. The Journal for Transdisciplinary Research in Southern Africa 16:1
Moon, Sung-Jae & Kyeong-Hwa Lee
2020. Deleuzian actualizations of the multiplicative concept: a study of perceptual flow and the transformation of learning assemblages. Educational Studies in Mathematics 104:2 ► pp. 221 ff.
Sinclair, Nathalie & Alf Coles
2020. Affect and Ritualisation in Early Number Work. In Theorizing and Measuring Affect in Mathematics Teaching and Learning, ► pp. 77 ff.
Streeck, Jürgen
Streeck, Jürgen
Abrahamson, Dor, Virginia J. Flood, Joshua A. Miele & Yue-Ting Siu
2019. Enactivism and ethnomethodological conversation analysis as tools for expanding Universal Design for Learning: the case of visually impaired mathematics students. ZDM 51:2 ► pp. 291 ff.
Karnam, DurgaPrasad, Harshit Agrawal, Pranay Parte, Saurabh Ranjan, Aniket Sule & Sanjay Chandrasekharan
2019. 2019 IEEE Tenth International Conference on Technology for Education (T4E), ► pp. 114 ff.
Mikulan, Petra & Nathalie Sinclair
2019. Stratigraphy as a method for studying the different modes of existence arising in the mathematical classroom. ZDM 51:2 ► pp. 239 ff.
Sinclair, Nathalie
2019. Role of the Senses in Mathematical Thinking and Learning. In Encyclopedia of Educational Philosophy and Theory, ► pp. 1 ff.
Sinclair, Nathalie
2024. Knowing as Remembering: Methodological Experiments in Embodied Experiences of Number. Digital Experiences in Mathematics Education 10:1 ► pp. 29 ff.
Sinclair, Nathalie & Elizabeth de Freitas
2019. Body studies in mathematics education: diverse scales of mattering. ZDM 51:2 ► pp. 227 ff.
Menz, Petra & Nathalie Sinclair
2018. Diagramming and Gesturing During Mathematizing: Kinesthetic and Haptic Interactions Support Mathematical Ideation. In Signs of Signification [ICME-13 Monographs, ], ► pp. 315 ff.
Ng, Oi-Lam & Nathalie Sinclair
2018. Drawing in Space: Doing Mathematics with 3D Pens. In Uses of Technology in Primary and Secondary Mathematics Education [ICME-13 Monographs, ], ► pp. 301 ff.
Pimm, David
2018. On Number Language: A Commentary on Chapter 3. In Building the Foundation: Whole Numbers in the Primary Grades [New ICMI Study Series, ], ► pp. 71 ff.
Rahaman, Jeenath, Harshit Agrawal, Nisheeth Srivastava & Sanjay Chandrasekharan
2018. Recombinant Enaction: Manipulatives Generate New Procedures in the Imagination, by Extending and Recombining Action Spaces. Cognitive Science 42:2 ► pp. 370 ff.
Jackiw, Nicholas & Nathalie Sinclair
2017. TouchCounts and Gesture Design. In Frontiers in Pen and Touch [Human–Computer Interaction Series, ], ► pp. 51 ff.
Ng, Oi-Lam
2016. Comparing Calculus Communication across Static and Dynamic Environments Using a Multimodal Approach. Digital Experiences in Mathematics Education 2:2 ► pp. 115 ff.
Ng, Oi-Lam
2019. Examining Technology-Mediated Communication Using a Commognitive Lens: the Case of Touchscreen-Dragging in Dynamic Geometry Environments. International Journal of Science and Mathematics Education 17:6 ► pp. 1173 ff.
Sinclair, Nathalie, Maria G. Bartolini Bussi, Michael de Villiers, Keith Jones, Ulrich Kortenkamp, Allen Leung & Kay Owens
2016. Recent research on geometry education: an ICME-13 survey team report. ZDM 48:5 ► pp. 691 ff.
This list is based on CrossRef data as of 5 january 2025. Please note that it may not be complete. Sources presented here have been supplied by the respective publishers. Any errors therein should be reported to them.