Article published In:
GestureVol. 14:3 (2014) ► pp.351–374
The haptic nature of gesture
Rethinking gesture with new multitouch digital technologies
This paper is about re-thinking gesture in order to reckon with its material and haptic nature, especially in the current multitouch technology environment. This re-thinking of gesture returns to the principle of indexicality found in Peirce’s material semiotics, and develops this principle through the work of Gilles Châtelet and Gilles Deleuze around hand-eye relationships. Drawing on the work of Jürgen Streek, we propose and discuss the notion of the tangible gesture, in the context of mathematical explorations of young children with a multitouch iPad environment designed to promote counting on and with the fingers.
Keywords: technology, mathematics, the senses
References
Alibali, Martha & Allysa DiRusso
(1999) The function of gesture in learning to count: More than keeping track. Cognitive Development, 141, 37–56. 
Angel, Maria & Anna Gibbs
(2013) At the time of writing: Digital media, gestures, and handwriting. Electronic Book Review. Retrieved 22.09.2013. [URL]
Bartolini-Bussi, Mariolina & Mara Boni
(2003) Instruments for semiotic mediation in primary school classrooms. For the Learning of Mathematics, 23 (2), 12–19.
Châtelet, Gilles
([1993] 2000) Les enjeux du mobile. Paris: Seuil. English translation by Robert Shore & Muriel Zagha (2000): Figuring space: Philosophy, mathematics, and physics. Dordrecht: Kluwer, 2000.
(2010) L’enchantement du virtuel. Philosophie, physique, mathématique. Paris, France: Presses de l’École normale supérieure.
de Freitas, Elizabeth & Nathalie Sinclair
(2014) Mathematics and the body: Material entanglements in the classroom. New York: Cambridge University Press.
Deleuze, Gilles
(2003) Francis Bacon: The logic of sensation. Translated from the French by Daniel W. Smith. London, UK: Continuum.
Gibson, James J
(1966) The senses considered as perceptual systems. Boston: Houghton Mifflin.
Goodwin, Charles
(2007) Environmentally coupled gestures. In Susan D. Duncan, Justine Cassell, & Elena T. Levy (Eds.), Gesture and the dynamic dimension of language: Essays in honor of David McNeill (pp. 195–212). Amsterdam: John Benjamins.
Kendon, Adam
(2000) Language and gesture: Unity or duality? In David McNeill (Ed.), Language and gesture (pp. 47–63). Cambridge, UK: Cambridge University Press.
(2004) Gesture: Visible action as utterance. Cambridge, UK: Cambridge University Press.
Kita, Sotaro
(Ed.) (2003) Pointing: Where language, culture, and cognition meet. Mahwah, NJ: Erlbaum.
Maturana, Humberto & Francisco Varela
([1984] 1992) The tree of knowledge: Biological roots of human understanding. Boston, MA: Shambhala Publications.
McNeill, David
(1992) Hand and mind: What gestures reveal about thought. Chicago: University of Chicago Press.
(2005) Gesture and thought. Chicago: University of Chicago Press.
Mulder, Axel
(1996) Hand gestures for HCI. Technical Report, NSERC Hand Centered Studies of Human Movement project. Burnaby, BC, Canada: Simon Fraser University. Available: [URL]
Nemirovsky, Ricardo, Molly Kelton, & Bohdan Rhodehamel
(2012) Gesture and imagination: On the constitution and uses of phantasms. Gesture, 12 (2), 130–165.
Noel, Marie-Pascale
(2005) Finger gnosia: A predictor of numerical abilities in children? Child Neuropsychology, 51, 413–430.
Peirce, Charles Sanders
([1894] 1998) What is a sign? In Nathan Houser, et al. (Eds.), The essential Peirce, Vol. 21 (pp. 4–10). Bloomington, IN: Indiana University Press.
(1932)
Elements of logic
. In Charles Hartshorne & Paul Weiss (Eds.), Collected papers of Charles Sanders Peirce, Vol. 21. Cambridge: Harvard University Press.
(1994) Peirce on signs: Writings on semiotic, James Hoopes (Ed.). Chapel Hill, NC: University of North Carolina Press.
Radford, Luis
(2002) The seen, the spoken and the written: A semiotic approach to the problem of objectification of mathematical knowledge. For the Learning of Mathematics, 22 (2), 14–23.
Rasmussen, Chris, Michelle Stephan, & Karen Allen
(2004) Classroom mathematical practices and gesturing. Journal of Mathematical Behavior, 231, 301–323.
Reynolds, Fione J. & Robert A. Reeve
(2002) Gesture in collaborative mathematical problem-solving. Journal of Mathematical Behavior, 201, 447–460.
Robutti, Ornella & Francesca Ferrara
(2002) Approaching algebra through motion experiences. In Anne D. Cockburn & Elena Nardi (Eds.),
Proceedings of the 26th PME International Conference
, Vol. 11 (pp. 111–118).
Rossini, Nicla
(2012) Reinterpreting gesture as language: Language in action. Amsterdam: IOS Press.
Roth, Wolff-Michael
(2001) Gestures: Their role in teaching and learning. Review of Educational Research, 71 (3), 365–392.
Rotman, Brian
(1987) Signifying nothing: A semiotics of zero. Basingstoke: Macmillan.
(2008) Becoming beside ourselves: The alphabet, ghosts, and distributed human beings. Durham: Duke University Press.
Sinclair, Nathalie & Nicholas Jackiw
(2011) Touchcounts [Software application for the iPad]. Burnaby, BC: Tangible Learning Project.
Sinclair, Nathalie & David Pimm
(2014) Number’s subtle touch: Expanding finger gnosis in the era of multi-touch technologies. In Nicol Cynthia et al. (Eds.),
Proceedings of the Psychology of Mathematics Education Conference
. Vancouver, Canada, July 15–20, 2014, Vol. 51 (pp. 209–216).
Streeck, Jürgen
Villamor, Craig, Dan Willis, & Luke Wroblewski
Cited by
Cited by 40 other publications
Abrahamson, Dor, Virginia J. Flood, Joshua A. Miele & Yue-Ting Siu
2019. Enactivism and ethnomethodological conversation analysis as tools for expanding Universal Design for Learning: the case of visually impaired mathematics students. ZDM 51:2 ► pp. 291 ff. 
Abrahamson, Dor, Kimiko Ryokai & Justin Dimmel
2023. Learning Mathematics with Digital Resources: Reclaiming the Cognitive Role of Physical Movement. In Handbook of Digital Resources in Mathematics Education [Springer International Handbooks of Education, ], ► pp. 1 ff.
Bairral, Marcelo
2021. Interacciones, toques en pantalla y aprendizaje de cuadriláteros. Revista Venezolana de Investigación en Educación Matemática 1:2 ► pp. e202107 ff.
Bakos, Sandy & David Pimm
2020. Beginning to Multiply (with) Dynamic Digits: Fingers as Physical–Digital Hybrids. Digital Experiences in Mathematics Education 6:2 ► pp. 145 ff.
Billion, Lara Kristina
2021. Reconstruction of the Interpretation of Geometric Diagrams of Primary School Children Based on Actions on Various Materials -<br /> A Semiotic Perspective on Actions. International Electronic Journal of Mathematics Education 16:3 ► pp. em0650 ff.
de Freitas, Elizabeth & Nathalie Sinclair
2023. Landing Sites, Cities, and Nonplaces: Collaborating Across the Conference Circuit. Qualitative Inquiry 29:1 ► pp. 7 ff.
Forte, Magali
2021. Identités sociomatérielles et création d’histoires plurilingues et numériques. The Canadian Modern Language Review 77:4 ► pp. 332 ff.
Gerner, Alexander Matthias, Thiemo Breyer, Niklas Grouls & Johannes F. M. Schick
2024. Diachronic Perspectives on Embodiment and Technology: Gestures and Artefacts – An Introduction. In Diachronic Perspectives on Embodiment and Technology [Philosophy of Engineering and Technology, 46], ► pp. 1 ff.
Güneş, Canan
2021. A Quantitative Shift Towards Multiplicative Thinking. Digital Experiences in Mathematics Education 7:3 ► pp. 361 ff.
Jackiw, Nicholas & Nathalie Sinclair
2017. TouchCounts and Gesture Design. In Frontiers in Pen and Touch [Human–Computer Interaction Series, ], ► pp. 51 ff.
Janßen, Thomas, Estela Vallejo-Vargas, Angelika Bikner-Ahsbahs & David A. Reid
2020. Design and Investigation of a Touch Gesture for Dividing in a Virtual Manipulative Model for Equation-solving. Digital Experiences in Mathematics Education 6:2 ► pp. 166 ff.
Karnam, DurgaPrasad, Harshit Agrawal, Pranay Parte, Saurabh Ranjan, Aniket Sule & Sanjay Chandrasekharan
2019. 2019 IEEE Tenth International Conference on Technology for Education (T4E), ► pp. 114 ff.
Karnam, DurgaPrasad, Harshit Agrawal, Pranay Parte, Saurabh Ranjan, Priyanka Borar, Prasanna Prakash Kurup, Amose Jebin Joel, Pattamadai Sankaran Srinivasan, Uddhav Suryawanshi, Aniket Sule & Sanjay Chandrasekharan
2021. Touchy feely vectors: A compensatory design approach to support model‐based reasoning in developing country classrooms. Journal of Computer Assisted Learning 37:2 ► pp. 446 ff.
Kelleher, William
2020. Digital media, the body and agency in a South African education institution from the perspective of narrative research. The Journal for Transdisciplinary Research in Southern Africa 16:1
Menz, Petra & Nathalie Sinclair
2018. Diagramming and Gesturing During Mathematizing: Kinesthetic and Haptic Interactions Support Mathematical Ideation. In Signs of Signification [ICME-13 Monographs, ], ► pp. 315 ff.
Mikulan, Petra & Nathalie Sinclair
2019. Stratigraphy as a method for studying the different modes of existence arising in the mathematical classroom. ZDM 51:2 ► pp. 239 ff.
Moon, Sung-Jae & Kyeong-Hwa Lee
2020. Deleuzian actualizations of the multiplicative concept: a study of perceptual flow and the transformation of learning assemblages. Educational Studies in Mathematics 104:2 ► pp. 221 ff.
Ng, Oi-Lam
2016. Comparing Calculus Communication across Static and Dynamic Environments Using a Multimodal Approach. Digital Experiences in Mathematics Education 2:2 ► pp. 115 ff.
Ng, Oi-Lam
2019. Examining Technology-Mediated Communication Using a Commognitive Lens: the Case of Touchscreen-Dragging in Dynamic Geometry Environments. International Journal of Science and Mathematics Education 17:6 ► pp. 1173 ff.
Ng, Oi-Lam & Nathalie Sinclair
2018. Drawing in Space: Doing Mathematics with 3D Pens. In Uses of Technology in Primary and Secondary Mathematics Education [ICME-13 Monographs, ], ► pp. 301 ff.
Pande, Prajakt
2021. Learning and expertise with scientific external representations: an embodied and extended cognition model. Phenomenology and the Cognitive Sciences 20:3 ► pp. 463 ff.
Pimm, David
2018. On Number Language: A Commentary on Chapter 3. In Building the Foundation: Whole Numbers in the Primary Grades [New ICMI Study Series, ], ► pp. 71 ff.
Rahaman, Jeenath, Harshit Agrawal, Nisheeth Srivastava & Sanjay Chandrasekharan
2018. Recombinant Enaction: Manipulatives Generate New Procedures in the Imagination, by Extending and Recombining Action Spaces. Cognitive Science 42:2 ► pp. 370 ff.
Robutti, Ornella, Cristina Sabena, Christina Krause, Carlotta Soldano & Ferdinando Arzarello
2022. Gestures in Mathematics Thinking and Learning. In Handbook of Cognitive Mathematics, ► pp. 685 ff.
Robutti, Ornella, Cristina Sabena, Christina Krause, Carlotta Soldano & Ferdinando Arzarello
2022. Gestures in Mathematics Thinking and Learning. In Handbook of Cognitive Mathematics, ► pp. 1 ff.
Shvarts, Anna, Rosa Alberto, Arthur Bakker, Michiel Doorman & Paul Drijvers
2021. Embodied instrumentation in learning mathematics as the genesis of a body-artifact functional system. Educational Studies in Mathematics 107:3 ► pp. 447 ff.
Sinclair, Nathalie
2019. Role of the Senses in Mathematical Thinking and Learning. In Encyclopedia of Educational Philosophy and Theory, ► pp. 1 ff.
Sinclair, Nathalie
2024. Knowing as Remembering: Methodological Experiments in Embodied Experiences of Number. Digital Experiences in Mathematics Education 10:1 ► pp. 29 ff.
Sinclair, Nathalie, Maria G. Bartolini Bussi, Michael de Villiers, Keith Jones, Ulrich Kortenkamp, Allen Leung & Kay Owens
2016. Recent research on geometry education: an ICME-13 survey team report. ZDM 48:5 ► pp. 691 ff.
Sinclair, Nathalie & Alf Coles
2020. Affect and Ritualisation in Early Number Work. In Theorizing and Measuring Affect in Mathematics Teaching and Learning, ► pp. 77 ff.
Sinclair, Nathalie & Elizabeth de Freitas
2019. Body studies in mathematics education: diverse scales of mattering. ZDM 51:2 ► pp. 227 ff.
Sinclair, Nathalie, Mariam Haspekian, Ornella Robutti & Alison Clark-Wilson
2022. Revisiting Theories That Frame Research on Teaching Mathematics with Digital Technology. In The Mathematics Teacher in the Digital Era [Mathematics Education in the Digital Era, 16], ► pp. 391 ff.
Sinclair, Nathalie & Eva Jablonka
2022. Mathematics Learning: Structured Ways of Moving With. In The Palgrave Handbook of Embodiment and Learning, ► pp. 419 ff.
Sinha, Ravi, Hillary Swanson, Jody Clarke-Midura, Jessica F. Shumway, Victor R. Lee & Sanjay Chandrasekharan
2023. Proceedings of the 2023 Symposium on Learning, Design and Technology, ► pp. 1 ff.
Streeck, Jürgen
Streeck, Jürgen
Tancredi, Sofia, Rotem Abdu, Dor Abrahamson & Ramesh Balasubramaniam
2021. Modeling nonlinear dynamics of fluency development in an embodied-design mathematics learning environment with Recurrence Quantification Analysis. International Journal of Child-Computer Interaction 29 ► pp. 100297 ff.
Tucker, Stephen I. & Teri N. Johnson
2022. Developing number sense with Fingu: a preschooler’s embodied mathematics during interactions with a multi-touch digital game. Mathematics Education Research Journal 34:2 ► pp. 393 ff.
Tucker, Stephen I. & Patricia S. Moyer-Packenham
2021. Exploring the phenomenon of distance in children’s interactions with touchscreen digital mathematics games. International Journal of Mathematical Education in Science and Technology 52:10 ► pp. 1447 ff.
Yeung, Wing-Leung & Oi-Lam Ng
2023. Characterizing touchscreen actions in technology-enhanced embodied learning for mathematics instruction in K-12 setting – A systematic review (2010–2023). Computers & Education 205 ► pp. 104881 ff.
This list is based on CrossRef data as of 30 march 2024. Please note that it may not be complete. Sources presented here have been supplied by the respective publishers. Any errors therein should be reported to them.