Article published in:
Interaction Studies
Vol. 22:1 (2021) ► pp. 110–140
Groups in human-agent interaction

Robots as an interactive-social medium in storytelling to multiple children

| Interaction Science Labs., ATR | Doshisha University
| Interaction Science Labs., ATR
| Interaction Science Labs., ATR | Keio University
| Interaction Science Labs., ATR | Doshisha University
| Doshisha University
| Interaction Science Labs., ATR
This paper investigates the effects of group interaction in a storytelling situation for children using two robots: a reader robot and a listener robot as a side-participant. We developed a storytelling system that consists of a reader robot, a listener robot, a display, a gaze model, a depth sensor, and a human operator who responds and provides easily understandable answers to the children’s questions. We experimentally investigated the effects of using a listener robot and either one or two children during a storytelling situation on the children’s preferences and their speech activities. Our experimental results showed that the children preferred storytelling with the listener robot. Although two children obviously produced more speech than one child, the listener robot discouraged the children’s speech regardless of whether one or two were listening.
Keywords: storytelling, child-robot interaction, multiple robots, group interaction
Article outline
  • 1.Introduction
  • 2.Related work
    • 2.1Robot as an information-providing medium
    • 2.2Interactive storytelling-robot systems for children
  • 3.System overview
    • 3.1Human-tracking system
    • 3.2Gaze model
    • 3.3Robot
    • 3.4Details of implemented behavior
    • 3.5Operator
  • 4.Experiment
    • 4.1Hypotheses and predictions about children’s speech activities and the number of robots
    • 4.2Participants
    • 4.3Conditions
    • 4.4Environment
    • 4.5Procedure
    • 4.6Measurements
  • 5.Results
    • 5.1Observations
    • 5.2Verification of prediction: Speech activities based on the number of children and robots
    • 5.3Additional analysis of subjective impressions
    • 5.4Children’s behaviors analysis
      • 5.4.1Behavior category of children
      • 5.4.2Conversation topics of children
      • 5.4.3Touch behaviors between robots and children
  • 6.Discussion
    • 6.1Implications of experiment results
    • 6.2Design implications for application perspective
    • 6.3Toward (semi-) autonomous storytelling
    • 6.4Limitations
  • 7.Conclusion
  • Conflict of interest
  • Acknowledgements
  • References
Published online: 17 September 2021
https://doi.org/10.1075/is.18033.tam
References
Alborzi, H., A. Druin, J. Montemayor, M. Platner, J. Porteous, L. Sherman, A. Boltman, G. Taxén, J. Best, and J. Hammer
(2000) “Designing StoryRooms: interactive storytelling spaces for children,” in Proceedings of the 3rd conference on Designing interactive systems: processes, practices, methods, and techniques, pp. 95–104. Crossref
Arai, H., M. Kimoto, T. Iio, K. Shimohara, R. Matsumura, and M. Shiomi
(2019) “How Can Robot’s Gaze Ratio and Body Direction Show an Awareness of Priority to the People with whom it is Interacting?,” IEEE Robotics and Automation Letters, vol. 4, no. 4, pp. 3798–3805. CrossrefGoogle Scholar
Bers, M. U., and J. Cassell
(1998) “Interactive storytelling systems for children: Using technology to explore language and identity,” Journal of Interactive Learning Research, vol. 9, no. 2, pp. 183.Google Scholar
Clark, H. H.
(1996) Using Language: Cambridge University Press. CrossrefGoogle Scholar
Cohen, J.
(1960) “A coefficient of agreement for nominal scales,” Educational and psychological measurement, vol. 20, no. 1, pp. 37–46. CrossrefGoogle Scholar
Collins, F.
(1999) “The use of traditional storytelling in education to the learning of literacy skills,” Early Child Development and Care, vol. 152, no. 1, pp. 77–108. CrossrefGoogle Scholar
Dahlbäck, N., A. Jönsson, and L. Ahrenberg
(1993) “Wizard of Oz studies: why and how,” in Proceedings of the 1st international conference on Intelligent user interfaces, Orlando, Florida, USA, pp. 193–200. Crossref
Fails, J. A., A. Druin, and M. L. Guha
(2014) “Interactive storytelling: Interacting with people, environment, and technology,” International Journal of Arts and Technology, vol. 7, no. 1, pp. 112–124. CrossrefGoogle Scholar
Fourati, N., A. Richard, S. Caillou, N. Sabouret, J.-C. Martin, E. Chanoni, and C. Clavel
(2016) “Facial Expressions of Appraisals Displayed by a Virtual Storyteller for Children,” Intelligent Virtual Agents: 16th International Conference, IVA 2016, Los Angeles, CA, USA, September 20–23, 2016, Proceedings, D. Traum, W. Swartout, P. Khooshabeh et al., eds., pp. 234–244, Cham: Springer International Publishing. CrossrefGoogle Scholar
Fridin, M.
(2014) “Storytelling by a kindergarten social assistive robot: A tool for constructive learning in preschool education,” Computers & education, vol. 701, pp. 53–64. CrossrefGoogle Scholar
Gharbi, M., P. V. Paubel, A. Clodic, O. Carreras, R. Alami, and J. M. Cellier
(2015) “Toward a better understanding of the communication cues involved in a human-robot object transfer,” in Robot and Human Interactive Communication (RO-MAN), 2015 24th IEEE International Symposium on, pp. 319–324. Crossref
Glas, D. F., T. Kanda, H. Ishiguro, and N. Hagita
(2011) “Teleoperation of multiple social robots,” IEEE Transactions on Systems, Man, and Cybernetics-Part A: Systems and Humans, vol. 42, no. 3, pp. 530–544. CrossrefGoogle Scholar
Goffman, E.
(1981) Forms of talk: University of Pennsylvania Press.Google Scholar
Gordon, G., S. Spaulding, J. Kory Westlund, J. J. Lee, L. Plummer, M. Martinez, M. Das, and C. Breazeal
(2016) “Affective Personalization of a Social Robot Tutor for Children’s Second Language Skills,” in Thirtieth AAAI Conference on Artificial Intelligence.Google Scholar
Gross, H.-M., H. Boehme, C. Schroeter, S. Müller, A. König, E. Einhorn, C. Martin, M. Merten, and A. Bley
(2009) “TOOMAS: interactive shopping guide robots in everyday use-final implementation and experiences from long-term field trials,” in Intelligent Robots and Systems, 2009. IROS 2009. The IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 2005–2012. CrossrefGoogle Scholar
Hato, Y., S. Satake, T. Kanda, M. Imai, and N. Hagita
(2010) “Pointing to space: modeling of deictic interaction referring to regions,” in Human-Robot Interaction (HRI), 2010 5th ACM/IEEE International Conference on, pp. 301–308. Crossref
Huisman, G.
(2017) “Social Touch Technology: A Survey of Haptic Technology for Social Touch,” IEEE Transactions on Haptics, vol. 10, no. 3, pp. 391–408. CrossrefGoogle Scholar
Iio, T., Y. Yoshikawa, and H. Ishiguro
(2016) “Pre-scheduled Turn-Taking between Robots to Make Conversation Coherent,” in Proceedings of the Fourth International Conference on Human Agent Interaction, Biopolis, Singapore, pp. 19–25. Crossref
(2017) “Retaining Human-Robots Conversation: Comparing Single Robot to Multiple Robots in a Real Event,” Journal of Advanced Computational Intelligence Intelligent Informatics, vol. 21, no. 4, pp. 675–685. CrossrefGoogle Scholar
Isbell, R., J. Sobol, L. Lindauer, and A. Lowrance
(2004) “The effects of storytelling and story reading on the oral language complexity and story comprehension of young children,” Early childhood education journal, vol. 32, no. 3, pp. 157–163. CrossrefGoogle Scholar
Kanda, T., H. Ishiguro, T. Ono, M. Imai, and K. Mase
(2002) “Multi-robot cooperation for human-robot communication,” in Robot and Human Interactive Communication, 2002. Proceedings. 11th IEEE International Workshop on, pp. 271–276. Crossref
Karatas, N., S. Yoshikawa, P. R. S. De Silva, and M. Okada
(2015) “Namida: Multiparty conversation based driving agents in futuristic vehicle,” in International Conference on Human-Computer Interaction, pp. 198–207.
Kennedy, J., V. Lemaignan, C. Montassier, P. Lavalade, B. Irfan, F. Papadopoulos, E. Senft, and T. Belpaeme
(2017, “Child Speech Recognition in Human-Robot Interaction: Evaluations and Recommendations,” in Proceedings of the 2017 ACM/IEEE International Conference on Human-Robot Interaction, Vienna, Austria, pp. 82–90. Crossref
Komatsubara, T., M. Shiomi, T. Kanda, and H. Ishiguro
(2017) “Can Using Pointing Gestures Encourage Children to Ask Questions?,” International Journal of Social Robotics, vol. 10, no. 4, pp. 387–399. CrossrefGoogle Scholar
Komatsubara, T., M. Shiomi, T. Kanda, H. Ishiguro, and N. Hagita
(2014) “Can a social robot help children’s understanding of science in classrooms?,” in Proceedings of the second international conference on Human-agent interaction, Tsukuba, Japan, pp. 83–90. Crossref
Kory, J., and C. Breazeal
(2014) “Storytelling with robots: Learning companions for preschool children’s language development,” in Robot and Human Interactive Communication, 2014 RO-MAN: The 23rd IEEE International Symposium on, pp. 643–648.
Lee, M. K., S. Kiesler, and J. Forlizzi
(2010) “Receptionist or information kiosk: how do people talk with a robot?,” in Proceedings of the 2010 ACM conference on Computer supported cooperative work, Savannah, Georgia, USA, pp. 31–40. Crossref
Leite, I., M. McCoy, M. Lohani, D. Ullman, N. Salomons, C. Stokes, S. Rivers, and B. Scassellati
(2015) “Emotional Storytelling in the Classroom: Individual versus Group Interaction between Children and Robots,” in Proceedings of the Tenth Annual ACM/IEEE International Conference on Human-Robot Interaction, Portland, Oregon, USA, pp. 75–82. Crossref
(2017) “Narratives with robots: The impact of interaction context and individual Differences on story recall and emotional Understanding,” Frontiers in Robotics and AI, vol. 41, pp. 29. CrossrefGoogle Scholar
Lombard, M., J. Snyder-Duch, and C. C. Bracken
(2002) “Content analysis in mass communication: Assessment and reporting of intercoder reliability,” Human Communication Research, vol. 28, no. 4, pp. 587–604. CrossrefGoogle Scholar
Matsuyama, Y., I. Akiba, A. Saito, and T. Kobayashi
(2013) “A Four-Participant Group Facilitation Framework for Conversational Robots,” in Proceedings of the SIGDIAL 2013 Conference, pp. 284–293.
Matsumura, R., M. Shiomi, T. Miyashita, H. Ishiguro, and N. Hagita
(2014) “Who is Interacting With me?; Identification of an Interacting Person Through Playful Interaction With a Small Robot,” IEEE Transactions on Human-Machine Systems, vol. 44, no. 2, pp. 169–179. CrossrefGoogle Scholar
(2015) “What kind of floor am I standing on? Floor surface identification by a small humanoid robot through full-body motions,” Advanced Robotics. CrossrefGoogle Scholar
Mutlu, B., J. Forlizzi, and J. Hodgins
(2006) “A storytelling robot: Modeling and evaluation of human-like gaze behavior,” in The 6th IEEE-RAS international conference on Humanoid robots, pp. 518–523.
Mutlu, B., T. Kanda, J. Forlizzi, J. Hodgins, and H. Ishiguro
(2012) “Conversational gaze mechanisms for human-like robots,” ACM Transactions on Interactive Intelligent Systems (TiiS), vol. 1, no. 2, pp. 12.Google Scholar
Mutlu, B., T. Shiwa, T. Kanda, H. Ishiguro, and N. Hagita
(2009) “Footing in human-robot conversations: how robots might shape participant roles using gaze cues,” in Proceedings of the 4th ACM/IEEE international conference on Human robot interaction, pp. 61–68. Crossref
Nourbakhsh, I. R., C. Kunz, and T. Willeke
(2003) “The mobot museum robot installations: A five year experiment,” in IEEE/RSJ International Conference on Intelligent Robots and Systems, 2003.(IROS 2003). pp. 3636–3641.
Park, H. W., M. Gelsomini, J. J. Lee, and C. Breazeal
(2017) “Telling stories to robots: The effect of backchanneling on a child’s storytelling,” in Proceedings of the 2017 ACM/IEEE international conference on human-robot interaction, pp. 100–108. Crossref
Powers, A., S. Kiesler, S. Fussell, and C. Torrey
(2007) “Comparing a computer agent with a humanoid robot,” in Human-Robot Interaction (HRI), 2007 2nd ACM/IEEE International Conference on, pp. 145–152. Crossref
Roussou, M.
(2004) “Learning by doing and learning through play: an exploration of interactivity in virtual environments for children,” Computers in Entertainment (CIE), vol. 2, no. 1, pp. 10–10. CrossrefGoogle Scholar
Sakamoto, D., K. Hayashi, T. Kanda, M. Shiomi, S. Koizumi, H. Ishiguro, T. Ogasawara, and N. Hagita
(2009) “Humanoid Robots as a Broadcasting Communication Medium in Open Public Spaces,” International Journal of Social Robotics, vol. 1, no. 2, pp. 157–169. CrossrefGoogle Scholar
Satake, S., K. Hayashi, K. Nakatani, and T. Kanda
(2015) “Field trial of an information-providing robot in a shopping mall,” in The IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2015) pp. 1832–1839. Crossref
Satake, S., T. Kanda, D. F. Glas, M. Imai, H. Ishiguro, and N. Hagita
(2013) “A robot that approaches pedestrians,” IEEE Transactions on Robotics, vol. 29, no. 2, pp. 508–524. CrossrefGoogle Scholar
Shi, C., M. Shiomi, T. Kanda, H. Ishiguro, and N. Hagita
(2015) “Measuring Communication Participation to Initiate Conversation in Human–Robot Interaction,” International Journal of Social Robotics, vol. 7, no. 5, pp. 889–910. CrossrefGoogle Scholar
Shi, C., M. Shiomi, C. Smith, T. Kanda, and H. Ishiguro
(2013) “A Model of Distributional Handing Interaction for a Mobile Robot,” in Robotics: Science and Systems.Google Scholar
Shinozawa, K., F. Naya, J. Yamato, and K. Kogure
(2005) “Differences in effect of robot and screen agent recommendations on human decision-making,” International Journal of Human-Computer Studies, vol. 62, no. 2, pp. 267–279. CrossrefGoogle Scholar
Shiomi, M., T. Kanda, I. Howley, K. Hayashi, and N. Hagita
(2015) “Can a Social Robot Stimulate Science Curiosity in Classrooms?,” International Journal of Social Robotics, vol. 7, no. 5, pp. 641–652. CrossrefGoogle Scholar
Shiomi, M., T. Kanda, H. Ishiguro, and N. Hagita
(2007) “Interactive Humanoid Robots for a Science Museum,” IEEE Intelligent Systems, no. 2, pp. 25–32. CrossrefGoogle Scholar
Shiomi, M., D. Sakamoto, T. Kanda, C. T. Ishi, H. Ishiguro, and N. Hagita
(2010) “Field Trial of a Networked Robot at a Train Station,” International Journal of Social Robotics, vol. 3, no. 1, pp. 27–40. CrossrefGoogle Scholar
Shiomi, M., K. Shinozawa, Y. Nakagawa, T. Miyashita, T. Sakamoto, T. Terakubo, H. Ishiguro, and N. Hagita
(2013) “Recommendation Effects of a Social Robot for Advertisement-Use Context in a Shopping Mall,” International Journal of Social Robotics, vol. 5, no. 2, pp. 251–262. CrossrefGoogle Scholar
Short, E. S., K. Swift-Spong, H. Shim, K. M. Wisniewski, D. K. Zak, S. Wu, E. Zelinski, and M. J. Matarić
(2017) “Understanding social interactions with socially assistive robotics in intergenerational family groups,” in Robot and Human Interactive Communication (RO-MAN), 2017 26th IEEE International Symposium on, pp. 236–241. Crossref
Tamura, Y., M. Kimoto, M. Shiomi, T. Iio, K. Shimohara, and N. Hagita
(2017) “Effects of a Listener Robot with Children in Storytelling,” in Proceedings of the 5th International Conference on Human Agent Interaction, Bielefeld, Germany, pp. 35–43. Crossref
Toh, L. P. E., A. Causo, P. W. Tzuo, I.-M. Chen, and S. H. Yeo
(2016) “A Review on the Use of Robots in Education and Young Children,” Educational Technology & Society, vol. 19, no. 2, pp. 148–163.Google Scholar
Van Gils, F.
(2005) “Potential applications of digital storytelling in education,” in 3rd Twente student conference on IT.
Vázquez, M., A. Steinfeld, S. E. Hudson, and J. Forlizzi
(2014) “Spatial and other social engagement cues in a child-robot interaction: Effects of a sidekick,” in Proceedings of the 2014 ACM/IEEE international conference on Human-robot interaction, pp. 391–398. Crossref
Wada, K., D. F. Glas, M. Shiomi, T. Kanda, H. Ishiguro, and N. Hagita
(2015) “Capturing Expertise: Developing Interaction Content for a Robot Through Teleoperation by Domain Experts,” International Journal of Social Robotics, vol. 7, no. 5, pp. 653–672. CrossrefGoogle Scholar
Yamaoka, F., T. Kanda, H. Ishiguro, and N. Hagita
(2010) “A model of proximity control for information-presenting robots,” IEEE Transactions on Robotics, vol. 26, no. 1, pp. 187–195. CrossrefGoogle Scholar
Yamazaki, A., K. Yamazaki, Y. Kuno, M. Burdelski, M. Kawashima, and H. Kuzuoka
(2008) “Precision timing in human-robot interaction: coordination of head movement and utterance,” in Proceeding of the twenty-sixth annual SIGCHI conference on Human factors in computing systems, Florence, Italy, pp. 131–140. Crossref
Yohanan, S., and K. E. MacLean
(2012) “The role of affective touch in human-robot interaction: Human intent and expectations in touching the haptic creature,” International Journal of Social Robotics, vol. 4, no. 2, pp. 163–180. CrossrefGoogle Scholar