Cover not available
Article published In:
Diagrammatic Reasoning
Edited by Riccardo Fusaroli and Kristian Tylén
[Pragmatics & Cognition 22:2] 2014
► pp. 206–223

Thinking in action

| Columbia Teachers College, Stanford University
| Stanford University

When thought overwhelms the mind, the mind uses the body and the world. Several studies reveal ways that people alone or together use gesture and marks on paper to structure and augment their thought for comprehension, inference, and discovery. The studies show that the mapping of thought to gesture or the page is more direct than the arbitrary mapping to language and suggest that these forms of visual/spatial/action representation are used to “translate” language into mental representations. It is argued that actions in space create patterns in the world that reflect abstractions, that the actions are incorporated into gestures and the patterns into diagrams, a network that integrates gesture, action, the designed world, and abstraction dubbed spraction.

Keywords: diagram, visual communication, gesture, language, thinking, action, design
Published online: 11 December 2015
DOI logo
https://doi.org/10.1075/pc.22.2.03tve
References
Andres, M., Seron, X., & Olivier, E
(2007) Contribution of hand motor circuits to counting. Journal of Cognitive Neuroscience, 19(4), 563–576. DOI logoGoogle Scholar
Bilda, Z. and Gero, J.S
(2006) Reasoning with internal and external representations: A case study with expert architects. In R. Sun and N. Miyake (Editors), Proceedings of the Cognitive Science Society, (pp. 1020–1026). Lawrence Erlbaum.Google Scholar
Braver, T.S., Cohen, J.D., Nystrom, L.E., Jonides, J., Smith, E.E., and Noll, D.C
(1997) A parametric study of prefrontal cortex involvement in human working memory. Neuroimage, 11, 49–62. DOI logoGoogle Scholar
Bruner, J.S
(1957) Going beyond the information given. NY: Norton.Google Scholar
(1966) On cognitive growth. In J.S. Bruner, R.R. Olver, & P.M. Greenfield (Eds.), Studies in cognitive growth. (pp. 1–29). Oxford, England: Wiley.Google Scholar
Donald, M
(1991) Origins of the modern mind. Cambridge: Harvard University Press.Google Scholar
Carlson, R.A., Avraamides, M.N., Cary, M., & Strasberg, S
(2007) What do the hands externalize in simple arithmetic? Journal of Experimental Psychology: Learning , Memory, and Cognition, 331, 747–756. DOI logoGoogle Scholar
Cartmill, E.A., Beilock, S.L., and Goldin-Meadow, S
(2012) A word in the hand: Human gesture links representations to actions. Philosophical Transactions of the Royal Society, B, 3671, 129–143. DOI logoGoogle Scholar
Clark, H.H
(1996) Using language. Cambridge, England: Cambridge University Press. DOI logoGoogle Scholar
Chu, M., & Kita, S
(2008) Spontaneous gestures during mental rotation tasks: Insights into the microdevelopment of the motor strategy. Journal of Experimental Psychology: General, 1371, 706–723. DOI logoGoogle Scholar
Duncker, K
(1945) On problem solving. Psychological Monographs, 68(5), 270.Google Scholar
Eisenegger, C., Herwig, U., & Jäncke, L
(2007) The involvement of primary motor cortex in mental rotation revealed by transcranial magnetic stimulation. European Journal of Neuroscience, 41, 1240–1244. DOI logoGoogle Scholar
Emmorey, K., Tversky, B., & Taylor, H.A
(2000) Using space to describe space: Perspective in speech, sign, and gesture. Journal of Spatial Cognition and Computation, 21, 157–180. DOI logoGoogle Scholar
Engle, R.A
(1998) Not channels but composite signals: Speech, gesture, diagrams and object demonstrations are integrated in multimodal explanations. In M.A. Gernsbacher & S.J. Derry (Eds.) Proceedings of the Twentieth Annual Conference of the Cognitive Science Society. Mahwah, NJ: Erlbaum.Google Scholar
Fitch, W.T., Hauser, M.D., and Chomsky, N
(2005) The evolution of the language faculty: Clarifications and implications. Cognition, 971, 179–210. DOI logoGoogle Scholar
Ganis, G., Keenan, J.P., Kosslyn, S.M., & Pascual-Leone, A
(2000) Transcranial magnetic stimulation of primary motor cortex affects mental rotation. Cerebral Cortex, 101, 175–180. DOI logoGoogle Scholar
Gentner, D. and Stevens, A.L
(Editors) (1983) Mental models. Erlbaum: Hillsdale, NJ.Google Scholar
Goldin-Meadow, S
(2003) Hearing gesture: How our hands help us think. Cambridge, MA: Harvard University Press.Google Scholar
Goldin-Meadow, S., Cook, S.W., & Mitchell, Z.A
(2009) Gesturing gives children new ideas about math. Psychological Science, 201, 267– 272. DOI logoGoogle Scholar
Gondry, M
(2013) Is the man who is tall happy? An animated conversation with Noam Chomsky. Partizan Films.Google Scholar
Heiser, J. and Tversky, B
(2006) Arrows in comprehending and producing mechanical diagrams. Cognitive Science, 301, 581–592. DOI logoGoogle Scholar
Heiser, J., Tversky, B., & Silverman, M
(2004) Sketches for and from collaboration. In J.S. Gero, B. Tversky, & T. Knight (Eds.), Visual and spatial reasoning in design III (pp. 69–78). Sydney: Key Centre for Design Research.Google Scholar
Hoesttetter, A.B. and Alibali, M.W
(2008) Visible embodiment: Gestures as simulated action. Psychonomic Bulletin and Review, 151, 495–514. DOI logoGoogle Scholar
Holyoak, K.J
(1994) Problem solving. In E.E. Smith & D.N. Osherson (Eds.), An invitation to cognitive science: Thinking (pp. 267–296). Cambridge: MIT Press.Google Scholar
Hutchins, E
(1995) Cognition in the wild. Cambridge, MA: MIT Press.Google Scholar
Iverson, J.M
(1999) How to get to the cafeteria: Gesture and speech in blind and sighted children’s spatial descriptions. Developmental Psychology, 351, 1132–1142. DOI logoGoogle Scholar
Jamalian, A., Giardino, V., and Tversky, B
(2013) Gestures for thinking. In M. Knauff, M. Pauen, N. Sabaenz, and I. Wachsmuth (Editors) Proceedings of the 35th Annual Conference of the Cognitive Science Society. Austin, TX: Cognitive Science Society.Google Scholar
Jamalian, A. and Tversky, B
(2012) Gestures alter thinking about time. In N. Miyake, D. Peebles, & R.P. Cooper (Eds.), Proceedings of the 34th Annual Conference of the Cognitive Science Society (pp. 551–557). Austin TX: Cognitive Science Society.Google Scholar
Johnson-Laird, P
(1983) Mental models. Cambridge: Harvard University Press.Google Scholar
Kansaku, K., Carver, B., Johnson, A., Matsuda, K, Sadato, N., and Hallett, M
(2007) The role of the human ventral premotor cortex in counting successive stimuli. Experimental Brain Research, 1781, 339–350. DOI logoGoogle Scholar
Kang, S., Tversky, B., and Black, J.B
2015). Coordinating gesture, word, and diagram: Explanations for adult experts and young novices. Spatial Cognition and Computation, 151, 1–26. DOI: 10.1080/13875868.2014.958837. DOI logo
(2012) From hands to minds: How gestures promote action understanding. In N. Miyake, D. Peebles, & R.P. Cooper (Eds.), Proceedings of the 34th Annual Conference of the Cognitive Science Society (pp. 551–557). Austin TX: Cognitive Science Society.Google Scholar
Kantrowitz, A
(2014) A cognitive ethnographic study of improvisational drawing by eight contemporary artists. Doctoral dissertation, Columbia Teachers College.
Kendon, A
(2004) Gesture: visible action as utterance. Cambridge: Cambridge University Press. DOI logoGoogle Scholar
Kessell, A.M. and Tversky, B
(2006) Using gestures and diagrams to think and talk about insight problems. In R. Sun and N. Miyake (Editors), Proceedings of the Meetings of the Cognitive Science Society. Mahwah, NJ: Erlbaum.Google Scholar
(2011) Visualizing space, time, and agents: Production, performance, and preference. Cognitive Processing, 121, 43–52. DOI logoGoogle Scholar
Kintsch, W
(1996) Comprehension: A paradigm for cognition. Cambridge: Cambridge University Press.Google Scholar
Kirsh, D
(1995) The intelligent use of space. Artificial Intelligence, 731, 31–68. DOI logoGoogle Scholar
Kosslyn, S.M
(1996) Image and brain: The resolution of the imagery debate. Cambridge: MIT Press.Google Scholar
Krauss, R.M., Chen, Y., and Gottesman, R.F
(2000) Lexical gestures and lexical access: A process model. In D. McNeill (Ed.), Language and gesture (pp. 261–283). New York: Cambridge University Press. DOI logoGoogle Scholar
Manoach, D.S., Schlaug, G., Siewert, B., Darby, D.G., Bly, B.M., Benfield, A., Edelman, R.R., and Warach, S
(1997) Prefrontal cortex fMRI signal changes are correlated with working memory load. Neuroreport, 81, 545–549. DOI logoGoogle Scholar
Maier, N.R.F
(1931) Reasoning in humans. II. The solution of a problem and its appearance in consciousness. Journal of Comparative Psychology, 121, 181–194. DOI logoGoogle Scholar
McNeill, D
(1992) Hand and mind. Chicago: University of Chicago Press.Google Scholar
(2005) Gesture and Thought. Chicago: University of Chicago Press. DOI logoGoogle Scholar
Netz, R
(1999) Linguistic formulae as cognitive tools. Pragmatics and Cognition, 71, 147–176. DOI logoGoogle Scholar
Newall, A., Shaw, J.C., and Simon, H.A
(1958) Elements of a theory of human problem solving. Psychological Review, 651, 151–166. DOI logoGoogle Scholar
Norman, D.A
(1993) Things that make us smart. Boston: Addison-Wesley Longman.Google Scholar
Rauscher, F.H., Krauss, R.M., & Chen, Y
(1996) Gesture, speech, and lexical access: The role of lexical movements in the processing of speech. Pyschological Science, 7(4), 226–231.Google Scholar
Schon, D.A
(1983) The reflective practitioner. NY: Harper Collins.Google Scholar
Schwartz, D.L., & Black, J.B
(1996) Shuttling between depictive models and abstract rules: Induction and fallback. Cognitive Science, 201, 457– 497. DOI logoGoogle Scholar
Shepard, R.N
(1984) Ecological constraints on internal representation: Resonant kinematics of perceiving, imagining, thinking, and dreaming. Psychological Review, 911, 417–447. DOI logoGoogle Scholar
Stjernfelt, F
(2011) Diagrammatology: An investigation on the borderlines of phenomenology, ontology, and semantics. NY: Springer.Google Scholar
Suchman, L
(1987) Plans and situated actions. Cambridge: Cambridge University Press.Google Scholar
Suwa, M. & Tversky, B
(1996) What architects see in their sketches: Implications for design tools. Human factors in computing systems: Conference companion (pp. 191–192). NY: ACM.Google Scholar
Suwa, M., Tversky, B., Gero, J., and Purcell, T
(2001) Seeing into sketches: Regrouping parts encourages new interpretations. In J.S. Gero, B. Tversky, and T. Purcell (Editors). Visual and spatial reasoning in design (pp. 207–219). Sydney, Australia: Key Centre of Design Computing and Cognition.Google Scholar
Suwa, M. & Tversky, B
(2003) Constructive perception: A skill for coordinating perception and conception. In R. Alterman and D. Kirsh (Editors), Proceedings of the Cognitive Science Society Meetings. Mahwah, NJ: Erlbaum.Google Scholar
Talmy, L
(1983) How language structures space. In H.L. Pick, Jr. & L.P. Acredolo (Eds.), Spatial orientation: Theory, research and application (pp. 225–282). N. Y.: Plenum. DOI logoGoogle Scholar
Taylor, H.A., & Tversky, B
(1992) Spatial mental models derived from survey and route descriptions. Journal of Memory and Language, 311, 261–282. DOI logoGoogle Scholar
Tversky, B., Heiser, J., Lee, P. and Daniel, M.-P
(2009) Explanations in gesture, diagram, and word. In K.R. Coventry, T. Tenbrink, & J.A. Bateman (Editors), Spatial Language and dialogue (pp. 119–131). Oxford: Oxford University Press. DOI logoGoogle Scholar
Tversky, B., & Lee, P.U
(1999) Pictorial and verbal tools for conveying routes. In Freksa, C., & Mark, D.M. (Eds.), Spatial information theory: cognitive and computational foundations of geographic information science (pp. 51–64). Berlin: Springer.Google Scholar
Tversky, B., Morrison, J.B. & Betrancourt, M
(2002) Animation: Can it facilitate? International Journal of Human Computer Studies. International Journal of Human Computer Studies, 571, 247–262. DOI logoGoogle Scholar
Tversky, B. and Suwa, M
(2009) Thinking with sketches. In A. Markman (Editor), Tools for innovation. Oxford: Oxford University Press. DOI logoGoogle Scholar
Tversky, B, Zacks, J., Lee, P.U., & Heiser, J
(2000) Lines, blobs, crosses, and arrows: Diagrammatic communication with schematic figures. In M. Anderson, P. Cheng, and V. Haarslev (Editors), Theory and application of diagrams (pp. 221–230). Berlin: Springer. DOI logoGoogle Scholar
Vygotsky, L
(1962) Thought and language. Cambridge: MIT Press. DOI logoGoogle Scholar
Wexler, M., Kosslyn, S.M., & Berthoz, A
(1998) Motor processes in mental rotation. Cognition, 681, 77–94. DOI logoGoogle Scholar
Wohlschläger, A., & Wohlschläger, A
(1998) Mental and manual rotatation. Journal of Experimental Psychology: Human Perception and Performance, 241, 397–412. DOI logoGoogle Scholar
Zacks, J., & Tversky, B
(1999) Bars and lines: A study of graphic communication. Memory and Cognition, 271, 1073–1079. DOI logoGoogle Scholar
Zhang, J-J. and Norman, D.A
(1994) Representations in distributed cognitive tasks. Cognitive Science, 181, 87–122. DOI logoGoogle Scholar
Cited by

Cited by 12 other publications

Arnulf, Jan Ketil
2020. Wittgenstein’s Revenge: How Semantic Algorithms Can Help Survey Research Escape Smedslund’s Labyrinth. In Respect for Thought [Theory and History in the Human and Social Sciences, ], ► pp. 285 ff. DOI logo
Austin, Elizabeth E. & Naomi Sweller
2017. Getting to the elephants: Gesture and preschoolers’ comprehension of route direction information. Journal of Experimental Child Psychology 163 ► pp. 1 ff. DOI logo
Fusaroli, Riccardo & Kristian Tylén
2014. Diagrammatic reasoning. Pragmatics & Cognition 22:2 ► pp. 183 ff. DOI logo
Koskela, Lauri & Ehud Kroll
2019. Model-Based Abduction in Design. In Design Computing and Cognition '18, ► pp. 247 ff. DOI logo
Koskela, Lauri & Ehud Kroll
2020. Demonstration, extension, and refinement of the re-proposed notion of design abduction. Artificial Intelligence for Engineering Design, Analysis and Manufacturing 34:2 ► pp. 286 ff. DOI logo
Law, Matthew V., Nnamdi Nwagwu, Amritansh Kwatra, Seo-Young Lee, Daniel M. Diangelis, Naifang Yu, Gonzalo Gonzalez-Pumariega, Amit Rajesh & Guy Hoffman
2024. Affinity Diagramming with a Robot. ACM Transactions on Human-Robot Interaction 13:1 ► pp. 1 ff. DOI logo
Lorini, Giuseppe & Stefano Moroni
2020. How to make norms with drawings: An investigation of normativity beyond the realm of words. Semiotica 2020:233 ► pp. 55 ff. DOI logo
Stevens, Michael Paul & Simon Harrison
2017. Spectrums of thought in gesture. Pragmatics & Cognition 24:3 ► pp. 441 ff. DOI logo
Tversky, Barbara
2017. Gestures can create diagrams (that are neither imagistic nor analog). Behavioral and Brain Sciences 40 DOI logo
Tversky, Barbara & Azadeh Jamalian
2021. Thinking Tools: Gestures Change Thought About Time. Topics in Cognitive Science 13:4 ► pp. 750 ff. DOI logo
Wang, Qi, Holly A Taylor & Tad T Brunyé
2019. East is not right: Spatial compatibility differs between egocentric and cardinal retrieval. Quarterly Journal of Experimental Psychology 72:5 ► pp. 1250 ff. DOI logo
Zheng, Chen & Barbara Tversky
2024. Putting it Together, Together. Cognitive Science 48:2 DOI logo

This list is based on CrossRef data as of 12 april 2024. Please note that it may not be complete. Sources presented here have been supplied by the respective publishers. Any errors therein should be reported to them.