Cover not available

Article In: Interaction Studies: Online-First Articles

Comparing a tutor and peer-learner social robot in a vocabulary learning activity

ORCiD logo with link | University of Waterloo
 | University of Waterloo
 | University of Waterloo
This content is being prepared for publication; it may be subject to changes.
Abstract
The purpose of this study is to compare two different roles for a social robot to assume in a robot-assisted learning activity. Study participants learned 12 vocabulary words in Esperanto with either a robot assuming a tutor role, or a peer-learner role. Results found that participants’ general attitudes and perceptions towards both robot roles were positive. Usability scores were high in both conditions, mostly positive user experiences in were reported both conditions. While there was no significant difference in learning outcomes between conditions, participants in the peer-learner condition completed the post-lesson quiz 30% faster, which could indicate more meaningful learning occurred and suggest that interaction and instructional characteristics associated with peer learning, such as increased engagement, confidence, or retrieval practice, contributed to this difference. A thematic analysis of participants’ responses to open-ended questions shows participants’ opinions on the system in both conditions, including a desire for more interaction with the peer-learner robot, feeling that the robot provides companionship and is interactive, feeling comfortable and not-judged by the robot, and envisioning real-life applications of such learning system. Findings from the thematic analysis also provide directions for future research and technology development for Robot-Assisted Language Learning (RALL).
Article outline
  • 1.Introduction and Motivation
  • 2.Related work
    • 2.1Robot roles in education
    • 2.2Robot-Assisted Language Learning
  • 3.Research question and hypotheses
  • 4.Methodology and design
    • 4.1Robot used
    • 4.2Robot movement design
    • 4.3Learning interface webpage application
    • 4.4Study recruitment
    • 4.5Experiment procedure, design and materials
      • 4.5.1Study procedure and setup
      • 4.5.2Experiment conditions
      • 4.5.3Tutor Robot condition
      • 4.5.4Peer-learner robot condition
      • 4.5.5Language selection
      • 4.5.6Lesson plan (experiment)
    • 4.6Manipulation Check
  • 5.Measures
    • 5.1Pre-experiment measures: Participant Demographics
    • 5.2Post-experiment measures: Participant Experience and Performance Assessment
      • 5.2.1Participant Experience
      • 5.2.2Esperanto Quiz
    • 5.3Data Analysis Methods
      • 5.3.1Statistical Analysis
      • 5.3.2Thematic Analysis
  • 6.Results
    • 6.1Quantitative results
      • 6.1.1Participant demographics
      • 6.1.2Participant personality traits
      • 6.1.3RoSAS social attribute scales
      • 6.1.4UEQ scales
      • 6.1.5SUS usability scores
      • 6.1.6Quiz results
    • 6.2Qualitative results
      • 6.2.1Thematic Codes
      • 6.2.2ICR agreement
    • 6.3Thematic analysis results
      • 6.3.1Theme 1: Face, movement and voice of robot
        • Subtheme 1.1 Face of robot
        • Subtheme 1.2 Movements of robot
        • Subtheme 1.3 Voice of Robot
      • 6.3.2Theme 2: Interaction with Robot
        • Subtheme 2.1 Perception of Robot behavior
        • Subtheme 2.2 Robot awareness, capability, and effectiveness
        • Subtheme 2.3 Desire for more interaction opportunities
        • Subtheme 2.4 Engagement with Robot
      • 6.3.3Theme 3: Overall system/technology
        • Subtheme 3.1 Technology acceptance
        • Subtheme 3.2 Robot role and contribution
      • 6.3.4Theme 4: Esperanto lesson
        • Subtheme 4.1 Pronunciation discrepancy and inaccuracy
        • Subtheme 4.2 Learning experience
        • Subtheme 4.3 Add customization and expand scope
  • 7.Discussion
    • 7.1Revisiting hypotheses
      • H1 Participants will have higher quiz scores when robot assumes peer-learner role
      • H2 Participants will take less time to complete the quiz when the robot assumes peer-learner role
      • H3 Participants will perceive the robot in a peer-learner role more positively (e.g., higher warmth, lower discomfort, higher competence)
      • H4 Robot assuming peer-learner role will yield higher ratings in usability and user experience from participants
    • 7.2Implications of Robot’s Role
  • 8.Limitations and future work
  • 9.Conclusion
  • Declarations
    • 9.2Ethics approval
    • 9.3Data availability
    • 9.4Materials availability
    • 9.5Code availability
    • 9.6Author contribution
  • Author queries
  • References
References (54)
References
Anderson–darling test. (2008). In The concise encyclopedia of statistics (pp. 12–14). New York, NY: Springer New York.Google Scholar logo with link to Google Scholar
Ausubel, D. P. (2000). The nature of meaning and meaningful learning. In The acquisition and retention of knowledge: A cognitive view (pp. 67–100). Dordrecht: Springer Netherlands. Google Scholar logo with link to Google Scholar
Bartneck, C., Belpaeme, T., Eyssel, F., Kanda, T., Keijsers, M., Sabanovic, S. (2020). Human-robot interaction: An introduction. Cambridge University Press. Google Scholar logo with link to Google Scholar
Baxter, P., Ashurst, E., Read, R., Kennedy, J., Belpaeme, T. (2017, 05). Robot education peers in a situated primary school study: Personalisation promotes child learning. PLOS ONE, 121, 1–23. Google Scholar logo with link to Google Scholar
Belpaeme, T., Kennedy, J., Ramachandran, A., Scassellati, B., Tanaka, F. (2018). Social robots for education: A review. Science Robotics, 3 (21), eaat5954, Google Scholar logo with link to Google Scholar
Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3 (2), 77–101, Google Scholar logo with link to Google Scholar
Britannica, E. (n.d.). Esperanto. Retrieved from [URL] (Accessed 2023-11-22)
Brooke, J. (1996). Sus: A quick and dirty usability scale. In (Vol. 1941, p. 189–194). London: Taylor and Francis.Google Scholar logo with link to Google Scholar
Cabeza, R., Kapur, S., Craik, F., McIntosh, A., Houle, S., Tulving, E. (1997, 03). Functional neuroanatomy of recall and recognition: A pet study of episodic memory. Journal of Cognitive Neuroscience, 91, 254–265. Google Scholar logo with link to Google Scholar
Carpinella, C. M., Wyman, A. B., Perez, M. A., Stroessner, S. J. (2017). The robotic social attributes scale (rosas): Development and validation. 2017 12th acm/ieee international conference on human-robot interaction (hri (p. 254–262).
Dautenhahn, K., Nehaniv, C., Walters, M., Robins, B., Kose, H., Mirza, N., Blow, M. (2009, 01). Kaspar – a minimally expressive humanoid robot for human–robot interaction research. Applied Bionics and Biomechanics, 61, 369–397. Google Scholar logo with link to Google Scholar
Duff, P. A. (2000). Repetition in foreign language classroom interaction. Second and foreign language learning through classroom interaction (pp. 109–138). Mahwah, NJ: Erlbaum.Google Scholar logo with link to Google Scholar
Engwall, O., & Lopes, J. (2022). Interaction and collaboration in robot-assisted language learning for adults. Computer Assisted Language Learning, 35 (5–6), 1273–1309, Google Scholar logo with link to Google Scholar
Feil-Seifer, D., & Mataric, M. (2009, 04). Human-robot interaction. In (p. 4643–4659). Springer, New York, NY.Google Scholar logo with link to Google Scholar
Ghiglino, D., Chevalier, P., Floris, F., Priolo, T., Wykowska, A. (2021). Follow the white robot: Efficacy of robot-assistive training for children with autism spectrum disorder. Research in Autism Spectrum Disorders, 861, 101822, Retrieved from [URL]
Gosling, S. D., Rentfrow, P. J., Swann, W. B. (2003). A very brief measure of the big-five personality domains. Journal of Research in Personality, 37 (6), 504–528, Retrieved from [URL]
Habibzadeh, F. (2017). Statistical data editing in scientific articles. J Korean Med Sci, 32 (7), 1072–1076, Retrieved from [URL]
Haynes, W. (2013). Bonferroni correction. In W. Dubitzky, O. Wolkenhauer, K.-H. Cho, & H. Yokota (Eds.), Encyclopedia of systems biology (pp. 154–154). New York, NY: Springer New York. Google Scholar logo with link to Google Scholar
Huang, C.-M., & Mutlu, B. (2013). Modeling and evaluating narrative gestures for humanlike robots. Robotics: Science and Systems IX1, ,Google Scholar logo with link to Google Scholar
Kanda, T., Hirano, T., Eaton, D., Ishiguro, H. (2004, 06). Interactive robots as social partners and peer tutors for children: A field trial. Human Computer Interaction (Special issues on human-robot interaction), 191, 61–84, Google Scholar logo with link to Google Scholar
Karpicke, J. D., & Grimaldi, P. J. (2012). Retrieval-based learning: A perspective for enhancing meaningful learning. Educational Psychology Review, 24 (3), 401–418, Google Scholar logo with link to Google Scholar
Khosla, R., Nguyen, K., Chu, M.-T. (2015). Socially assistive robot enabled home-based care for supporting people with autism. Pacis 2015 proceedings.Google Scholar logo with link to Google Scholar
Laugwitz, B., Held, T., Schrepp, M. (2008, 11). Construction and evaluation of a user experience questionnaire. (Vol. 52981, p. 63–76). Google Scholar logo with link to Google Scholar
Manzi, F., Peretti, G., Di Dio, C., Cangelosi, A., Itakura, S., Kanda, T., … Marchetti, A. (2020). A robot is not worth another: Exploring children’s mental state attribution to different humanoid robots. Frontiers in Psychology, Volume 111 — 2020, , Google Scholar logo with link to Google Scholar
Manzi, F., Peretti, G., Dio, C. D., Itakura, S., Kanda, T., Ishiguro, H., … Marchetti, A. (2025). Cross-cultural moral judgments and emotions in preschoolers: human and robot transgressions in japan and italy. Behaviour & Information Technology, 44 (14), 3484–3500, Google Scholar logo with link to Google Scholar
McHugh, M. L. (2012). Interrater reliability: The kappa statistic. Biochem Med (Zagreb), 22 (3), 276–282, Google Scholar logo with link to Google Scholar
McNeill, D. (1992). Guide to gesture classification. Hand and mind: What gestures reveal about thought (pp. 75–104). Chicago: University of Chicago Press. Retrieved from [URL]
Moore, P. G. (1957). The two-sample t-test based on range. Biometrika, 44 (3/4), 482–489, Retrieved 2023-12-27, from [URL]
Nádasdy, Á. (1993). Language families. E. Pásztor, J. Vajda, & F. Loew (Eds.), Language and speech (Vol. 561). Vienna: Springer. Google Scholar logo with link to Google Scholar
Open Source Robotics Foundation (2023a). Ros noetic ninjemys. [URL]. (Accessed: 2023-11-22)
(2023b). Ros — robot operating system. [URL]. (Accessed: 2023-11-22)
O’Connor, C., & Joffe, H. (2020). Intercoder reliability in qualitative research: Debates and practical guidelines. International Journal of Qualitative Methods, 191, 1609406919899220, Google Scholar logo with link to Google Scholar
Park, H. W., Rosenberg-Kima, R., Rosenberg, M., Gordon, G., Breazeal, C. (2017). Growing growth mindset with a social robot peer. Proceedings of the 2017 acm/ieee international conference on human-robot interaction (p. 137–145). New York, NY, USA: Association for Computing Machinery. Retrieved from
Peretti, G., Manzi, F., Di Dio, C., Cangelosi, A., Harris, P. L., Massaro, D., Marchetti, A. (2023). Can a robot lie? young children’s understanding of intentionality beneath false statements. Infant and Child Development, 32 (2), e2398, Google Scholar logo with link to Google Scholar
Qualtrics (2005–2024). Qualtrics. Available online. Retrieved from [URL] (Development Company: Qualtrics, Location: Provo, Utah, USA)
Rajki, A. (2007). An etymological dictionary of the esperanto language. Retrieved from [URL] (Accessed: 2024-11-28)
Randall, N. (2019, December). A survey of robot-assisted language learning (rall). ACM Transactions on Human-Robot Interaction, 9 (1), Article 7, Google Scholar logo with link to Google Scholar
Robins, B., & Dautenhahn, K. (2018). Kaspar, the social robot and ways it may help children with autism — an overview. Enfance, 11, 91–102, Google Scholar logo with link to Google Scholar
ROBOTIS (2023). Dynamixel sdk. [URL]. (Accessed: 2023-11-22)
(2023). Dynamixelsdk. [URL]. (Accessed: 2023-11-22)
(2023a). U2d2. [URL]. (Accessed: 2023-11-22)
(2023b). U2d2 power hub. [URL]. (Accessed: 2023-11-22)
ROBOTIS, INC. (2023). Robotis engineer kit 1 [us]. [URL]. 26228 Enterprise Ct., Lake Forest, CA 92630, United States of America. (Accessed: 2023-11-22)
Salomons, N., Pineda, K. T., Adéjàre, A., Scassellati, B. (2022). ”we make a great team!”: Adults with low prior domain knowledge learn more from a peer robot than a tutor robot. 2022 17th acm/ieee international conference on human-robot interaction (hri) (p. 176–184). Sapporo, Japan.
Schrepp, M., Hinderks, A., Thomaschewski, J. (2014, 06). Applying the user experience questionnaire (ueq) in different evaluation scenarios. (p. 383–392). Google Scholar logo with link to Google Scholar
Services, A. W. (n.d.). List of supported languages and voices. Retrieved from [URL] (Accessed 2023-11-22)
So, W.-C., Cheng, C.-H., Lam, W.-Y., Huang, Y., Ng, K.-C., Tung, H.-C., Wong, W. (2020, Feb 01). A robot-based play-drama intervention may improve the joint attention and functional play behaviors of chinese-speaking preschoolers with autism spectrum disorder: A pilot study. Journal of Autism and Developmental Disorders, 50 (2), 467–481, Retrieved from Google Scholar logo with link to Google Scholar
Spatola, N., & Wudarczyk, O. A. (2021). Ascribing emotions to robots: Explicit and implicit attribution of emotions and perceived robot anthropomorphism. Computers in Human Behavior, 1241, 106934, Google Scholar logo with link to Google Scholar
Ubuntu (Accessed 2023-12-19). Ubuntu Server 20.04. [URL]
Wikipedia (n.d.). Microsoft text-to-speech voices. Retrieved from [URL] (Accessed 2023-11-22)
Wiktionary Contributors (2024). Wiktionary, The Free Dictionary. Retrieved from [URL] (Accessed 2023-11-22)
Wong, K. P. (2015). Facilitating a meaningful learning experience for students by multimedia teaching approach.. Retrieved from [URL]
Wood, L. J., Dautenhahn, K., Rainer, A., Robins, B., Lehmann, H., Syrdal, D. S. (2013). Robot-mediated interviews — how effective is a humanoid robot as a tool for interviewing young children? PLOS ONE, 8 (3), 1–13, Google Scholar logo with link to Google Scholar
Zaga, C., Lohse, M., Truong, K. P., Evers, V. (2015). The effect of a robot’s social character on children’s task engagement: Peer versus tutor. A. Tapus, E. André, J.-C. Martin, F. Ferland, & M. Ammi (Eds.), Social robotics (pp. 704–713). Cham: Springer International Publishing. Google Scholar logo with link to Google Scholar
Mobile Menu Logo with link to supplementary files background Layer 1 prag Twitter_Logo_Blue