Cover not available
Article published In:
Gesture
Vol. 19:1 (2020) ► pp.128–155

Learning from an avatar video instructor

The role of gesture mimicry

| University of Wisconsin – Madison
| Indiana University
| University of Wisconsin – Madison
| University of Wisconsin – Madison

Teachers often produce gestures, and, in some cases, students mimic their teachers’ gestures and adopt them into their own repertoires. However, little research has explored the role of gesture mimicry in technology-based learning contexts. In this research, we examined variations in the rate and form of students’ gestures when learning from a computer-animated pedagogical avatar. Twenty-four middle school students received a lesson on polynomial multiplication from a gesturing avatar video instructor. After the lesson, students were asked to provide an explanation of what they learned. Students varied in their gesture rates, and some students produced gestures that were similar in form to the avatar’s gestures. Students who produced gestures that aligned with the teacher’s gestures scored higher than those who did not produce such gestures. These results suggest that middle school students’ gestures play a key role when learning a mathematics lesson from an avatar instructor.

Keywords: gesture mimicry, avatar instructor, algebra, instructional technology
Article outline
  • Method
    • Participants
    • Design and procedure
    • Materials
    • Pretest
    • Lesson
    • Explanation
    • Midtest
    • Posttest
    • Transfer test
    • Coding
      • Scoring
      • Gesture
  • Results
    • Pretest
    • Explanation
    • Learning from the lesson
  • Discussion
  • Acknowledgements
  • References
Published online: 18 May 2021
DOI logo
https://doi.org/10.1075/gest.18019.ves
References
Adamo-Villani, Nicoletta, Wilbur, Ronnie, Eccarius, Petra, & Abe-Harris, Laverne
(2009) Effects of character geometric model on the perception of sign language animation. Proceedings of the 2009 Second International Conference in Visualization (pp. 72–75). Washington, DC: IEEE Computer Society. DOI logoGoogle Scholar
Anasingaraju, Saikiran, Wu, Meng-Lin, Adamo-Villani, Nicoletta, Popescu, Voicu, Cook, Susan W., Nathan, Mitchell, & Alibali, Martha W.
(2016) Digital learning activities delivered by eloquent instructor avatars: Scaling with problem instance. In Proceedings of SIGGRAPH Asia 2016 Symposium on Education, Article 5, 1–7. New York, NY: ACM. DOI logoGoogle Scholar
Alibali, Martha W., Flevares, Lucia M., & Goldin-Meadow, Susan
(1997) Assessing knowledge conveyed in gesture: Do teachers have the upper hand? Journal of Educational Psychology, 891, 183–193. DOI logoGoogle Scholar
Alibali, Martha W., Spencer, Robert C., Knox, Lucy, & Kita, Sotaro
(2011) Spontaneous gestures influence strategy choices in problem solving. Psychological Science, 22 (9), 1138–1144. DOI logoGoogle Scholar
Baylor, Amy L.
(2003, July). The impact of three pedagogical agent roles. In Proceedings of the second international joint conference on Autonomous agents and multiagent systems (pp. 928–929). ACM. DOI logoGoogle Scholar
Berch, Denise, Singleton, Jenny L., & Perry, Michelle
(1995) Many hands make learning math easy. Paper presented at the biennial meeting of the Society for Research in Child Development, Indianapolis, IN.
Broaders, Sara C., Cook, Susan W., Mitchell, Zachary, & Goldin-Meadow, Susan
(2007) Making children gesture brings out implicit knowledge and leads to learning. Journal of Experimental Psychology: General, 136 (4), 539–550. DOI logoGoogle Scholar
Buisine, Stéphanie & Martin, Jean-Claude
(2007) The effects of speech–gesture cooperation in animated agents’ behavior in multimedia presentations. Interacting with Computers, 19 (4), 484–493. DOI logoGoogle Scholar
Cassell, Justine, Sullivan, Joseph, Prevost, Scott, & Churchill, Elizabeth
(2000) Embodied conversational agents. Cambridge, MA: MIT Press. DOI logoGoogle Scholar
Church, Ruth Breckinridge, Ayman-Nolley, Saba, & Mahootian, Shahrzad
(2004) The role of gesture in bilingual education: Does gesture enhance learning? International Journal of Bilingual Education and Bilingualism, 7 (4), 303–319. DOI logoGoogle Scholar
Church, Ruth Breckinridge & Goldin-Meadow, Susan
(1986) The mismatch between gesture and speech as an index of transitional knowledge. Cognition, 23 (1), 43–71. DOI logoGoogle Scholar
Cook, Susan W., Duffy, Ryan G., & Fenn, Kimberly M.
(2013) Consolidation and transfer of learning after observing hand gesture. Child Development, 84 (6), 1863–1871. DOI logoGoogle Scholar
Cook, Susan W., Friedman, Howard S., Duggan, Katherine A., Cui, Jian, & Popescu, Voicu
(2017) Hand gesture and mathematics learning: lessons from an Avatar. Cognitive Science, 41 (2), 518–535. DOI logoGoogle Scholar
Cook, Susan W. & Goldin-Meadow, Susan
(2006) The role of gesture in learning: Do children use their hands to change their minds? Journal of Cognition and Development, 7 (2), 211–232. DOI logoGoogle Scholar
Cook, Susan W., Mitchell, Zachary, & Goldin-Meadow, Susan
(2008) Gesturing makes learning last. Cognition, 106 (2), 1047–1058. DOI logoGoogle Scholar
Davis, Robert O.
(2018) The impact of pedagogical agent gesturing in multimedia learning environments: A meta-analysis. Educational Research Review, 241, 193–209. DOI logoGoogle Scholar
ELAN (Version 5.0.0-beta) [Computer software]
(2017, April 18). Nijmegen: Max Planck Institute for Psycholinguistics. Retrieved from [URL]
Ericsson, K. Anders & Simon, Herbert A.
(1993) Protocol analysis. Cambridge, MA: MIT press. DOI logoGoogle Scholar
Fyfe, Emily R., Alibali, Martha W., & Nathan, Mitchell
(2017) The promise and pitfalls of making connections in mathematics. In E. Galindo & J. Newton (Eds.), Proceedings of the 39th annual meeting of the North American Chapter of the International Group for the Psychology of Mathematics Education (pp. 717–724). Indianapolis, IN: Hoosier Association of Mathematics Teacher Educators.Google Scholar
Goldin-Meadow, Susan
(2005) Hearing gesture: How our hands help us think. Cambridge, MA: Harvard University Press.Google Scholar
Goldin-Meadow, Susan & Alibali, Martha W.
(2013) Gesture’s role in speaking, learning, and creating language. Annual Review of Psychology, 641, 8.1–8.27. DOI logoGoogle Scholar
Goldin-Meadow, Susan & Beilock, Sian L.
(2010) Action’s influence on thought: The case of gesture. Perspectives on Psychological Science, 5 (6), 664–674. DOI logoGoogle Scholar
Goldin-Meadow, Susan, Cook, Susan W., & Mitchell, Zachary A.
(2009) Gesturing gives children new ideas about math. Psychological Science, 20 (3), 267–272. DOI logoGoogle Scholar
Holmes, Jeffrey
(2007) Designing agents to support learning by explaining. Computers & Education, 48 (4), 523–547. DOI logoGoogle Scholar
Hostetter, Autumn B.
(2011) When do gestures communicate? A meta-analysis. Psychological Bulletin, 137 (2), 297. DOI logoGoogle Scholar
Kimbara, Irene
(2006) On gestural mimicry. Gesture, 6(1), 39–61. DOI logoGoogle Scholar
Kelly, Spencer D., Church, Ruth Breckinridge, & Alibali, Martha W.
(2017) Understanding gesture: Description, mechanism, function. In Ruth Breckinridge Church, Martha W. Alibali, & Spencer D. Kelly (Eds.), Why gesture? How the hands function in speaking, thinking, and communicating (pp. 3–10). Amsterdam: John Benjamins. DOI logoGoogle Scholar
Kontra, Carly, Goldin-Meadow, Susan, & Beilock, Sian L.
(2012) Embodied learning across the life span. Topics in Cognitive Science, 4 (4), 731–739. DOI logoGoogle Scholar
Lester, James C., Converse, Sharolyn A., Kahler, Susan E., Barlow, S. Todd, Stone, Brian A., & Bhogal, Ravinder S.
(1997, March). The persona effect: affective impact of animated pedagogical agents. In Proceedings of the ACM SIGCHI Conference on Human factors in computing systems (pp. 359–366). ACM. DOI logoGoogle Scholar
Libertus, Klaus & Needham, Amy
(2010) Teach to reach: The effects of active vs. passive reaching experiences on action and perception. Vision Research, 50 (24), 2750–2757. DOI logoGoogle Scholar
Louwerse, M. Max, Graesser, Arthur C., McNamara, Danielle S., & Lu, Shulan
(2009) Embodied conversational agents as conversational partners. Applied Cognitive Psychology, 231, 1244–1255. DOI logoGoogle Scholar
Lusk, Mary M. & Atkinson, Robert K.
(2007) Animated pedagogical agents: Does their degree of embodiment impact learning from static or animated worked examples? Applied Cognitive Psychology: The Official Journal of the Society for Applied Research in Memory and Cognition, 21 (6), 747–764. DOI logoGoogle Scholar
McNeill, David
(1992) Hand and mind: What gestures reveal about thought. Chicago: University of Chicago Press.Google Scholar
Moreno, Roxana & Mayer, Richard
(2007) Interactive multimodal learning environments. Educational Psychology Review, 19 (3), 309–326. DOI logoGoogle Scholar
Mori, Masahiro, MacDorman, Karl F., & Kageki, Norri
(2012) The uncanny valley [from the field]. IEEE Robotics & Automation Magazine, 19 (2), 98–100. DOI logoGoogle Scholar
Moses, Robert P. & Cobb, Charles E.
(2001) Radical equations: Math literacy and civil rights. Boston: Beacon Press.Google Scholar
Nathan, Mitchell J. & Walkington, Candace
(2017) Grounded and embodied mathematical cognition: Promoting mathematical insight and proof using action and language. Cognitive research: principles and implications, 2 (1), 9. DOI logoGoogle Scholar
NGACBP (National Governors Association Center for Best Practices) & Council of Chief State School Officers
(2010) Common Core state standards for mathematics. Washington, DC: Authors.Google Scholar
de Nooijer, Jacqueline A., Van Gog, Tamara, Paas, Fred, & Zwaan, Rolf A.
(2013) Effects of imitating gestures during encoding or during retrieval of novel verbs on children’s test performance. Acta Psychologica, 144 (1), 173–179. DOI logoGoogle Scholar
Noma, Tsukasa, Zhao, Liwei, & Badler, Norman I.
(2000) Design of a virtual human presenter. IEEE Computer Graphics Applications, 20 (4), 49–85. DOI logoGoogle Scholar
Parrill, Fey & Kimbara, Irene
(2006) Seeing and hearing double: The influence of mimicry in speech and gesture on observers. Journal of Nonverbal Behavior, 30 (4), 157–166. DOI logoGoogle Scholar
Ping, Raedy & Goldin-Meadow, Susan
(2008) Hands in the air: Using ungrounded iconic gestures to teach children conservation of quantity. Developmental Psychology, 44 (5), 1277–1287. DOI logoGoogle Scholar
Popescu, Voicu, Adamo-Villani, Nicoletta, Wu, Meng-Lin, Rajesakaran, Suren D., Alibali, Martha W., Nathan, Mitchell J., & Cook, Susan W.
(2014, April). Animation killed the video star. Paper presented at the CHI (Conference on Human Factors in Computing Systems), Satellite Workshop on Gesture-Based Interaction Design: Communication and Cognition . Toronto, Canada.
Singer, Melissa A. & Goldin-Meadow, Susan
(2005) Children learn when their teacher’s gestures and speech differ. Psychological Science, 16 (2), 85–89. DOI logoGoogle Scholar
Tellier, Marion
(2008) The effect of gestures on second language memorisation by young children. Gesture, 8 (2), 219–235. DOI logoGoogle Scholar
Valenzeno, Laura, Alibali, Martha W., & Klatzky, Roberta
(2003) Teachers’ gestures facilitate students’ learning: A lesson in symmetry. Contemporary Educational Psychology, 28 (2), 187–204. DOI logoGoogle Scholar
Yeo, Amelia, Cook, Susan W., Nathan, Mitchell J., Popescu, Voicu, & Alibali, Martha W.
(2018) Instructor gesture improves encoding of mathematical representations. In T. T. Rogers, M. Rau, X. Zhu, & C. W. Kalish (Eds.), Proceedings of the 40th Annual Conference of the Cognitive Science Society (pp. 2723–2728).Google Scholar