Effect of synchronous robot motion on human synchrony and enjoyment perception:

Meneses, Alexis; Yoshikawa, Yuichiro; Ishiguro, Hiroshi

doi:10.1075/is.18027.men

Article published In:

Interaction Studies
Vol. 22:1 (2021) ► pp.86–109

Groups in human-agent interaction

Effect of synchronous robot motion on human synchrony and enjoyment perception

Alexis Meneses | Osaka University / JST ERATO

Yuichiro Yoshikawa | Osaka University / JST ERATO

Hiroshi Ishiguro | Osaka University / JST ERATO

Enhancing synchronization among people when synchronization is lacking is believed to improve their social skills, learning processes, and proficiency in musical rhythmic development. Greater synchronization among people can be induced to improve the rhythmic interaction of a system with multiple dancing robots that dance to a drum beat. A series of experiments were conducted to examine the human–human synchrony between persons that participated in musical sessions with robots. In this study, we evaluated: (a) the effect of the number of robots on a subject’s ability to synchronize with an experimenter; (b) the effect of the type of robot synchrony, namely, whether the robots did or did not represent the subject’s rhythm; (c) the effect of an in-sync and out-of-sync robot on a subject’s behavior. We found that: (a) three robots increased the level of synchronization between the subject and experimenter and their enjoyment level; (b) robots may induce greater synchronization between the subject and experimenter by reproducing the rhythms of not only the experimenter but also of the subject compared to when only the experimenter’s rhythms had been reproduced; (c) the robots in-sync had greater influence on the natural rhythm of the subject.

Keywords: multiple robots, rhythm synchronization, inter-persons synchronization

Article outline

Introduction
Experiment 1
- Method
- Subjects
- Experimental setup
  - Humanoid robots
  - Rhythm detection system
- Environment
- Procedure
- Evaluation
- Results
Experiment 2
- Method
- Subjects
- Experimental setup
- Procedure
- Evaluation
Results
Experiment 3
- Method
- Subjects
- Experimental setup
- Environment
- Procedure
- Evaluation
Results
Discussion
Limitations
Conclusion
References

Published online: 17 September 2021

https://doi.org/10.1075/is.18027.men

References

Alimardani, M., Nishio, S., & Ishiguro, H.

(2013) Humanlike robot hands controlled by brain activity arouse illusion of ownership in operators. Scientific reports, 31, 2396.

(2014) Effect of biased feedback on motor imagery learning in BCI-teleoperation system. Frontiers in systems neuroscience, 81, 52.

Arimoto, T., Yoshikawa, Y., & Ishiguro, H.

(2018) Multiple-Robot Conversational Patterns for Concealing Incoherent Responses. International Journal of Social Robotics, 1–11.

Avrunin, E., Hart, J., Douglas, A., & Scassellati, B.

(2011, March). Effects related to synchrony and repertoire in perceptions of robot dance. In Proceedings of the 6th international conference on Human-robot interaction (pp. 93–100). ACM.

Barsade, S. G.

(2002) The ripple effect: Emotional contagion and its influence on group behavior. Administrative Science Quarterly, 47(4), 644–675.

Bhide, A., Power, A., & Goswami, U.

(2013) A rhythmic musical intervention for poor readers: A comparison of efficacy with a letter-based intervention. Mind, Brain, and Education, 7(2), 113–123.

Blakemore et al.

(1999) Spatio-Temporal Prediction Modulates the Perception of Self-Produced Stimuli, Journal of Cognitive Neuroscience, 11:5, pp.551–559.

Botvinick, M., & Cohen, J.

(1998) Rubber hands ‘feel’ touch that eyes see. Nature, 391(6669), 756.

Briggs, G., & Scheutz, M.

(2014) How robots can affect human behavior: Investigating the effects of robotic displays of protest and distress. International Journal of Social Robotics, 6(3), 343–355.

Brown, S.

(2000) Evolutionary models of music: From sexual selection to group selection. In Perspectives in ethology (pp. 231–281). Springer, Boston, MA.

Cohen, G. L.

(2003) Party over policy: The dominating impact of group influence on political beliefs. Journal of personality and social psychology, 85(5), 808.

Crick, C., Munz, M., & Scassellati, B.

(2006, September). Synchronization in social tasks: Robotic drumming. In Robot and Human Interactive Communication, 2006. ROMAN 2006. (pp. 97–102). The 15th IEEE International Symposium.

David, D., Wade-Woolley, L., Kirby, J., Smithrim, K.

(2007) Rhythm and reading development in school-age children: A longitudinal study. Journal of Research in Reading. Volume 30, Issue 2, 169–183.

Eerola, T., Luck, G., & Toiviainen, P.

(2006, August). An investigation of pre-schoolers’ corporeal synchronization with music. In Proceedings of the 9th international conference on music perception and cognition (pp. 472–476). The Society for Music Perception and Cognition and European Society for the Cognitive Sciences of Music Bologna.

Gan, L., Huang, Y., Zhou, L., Qian, C., & Wu, X.

(2015) Synchronization to a bouncing ball with a realistic motion trajectory. Scientific reports, 51, 11974.

Hoffman, G., & Vanunu, K.

(2013) Effects of Robotic Companionship on Music Enjoyment and Agent Perception. In Proceedings of the 3rd ACM/IEEE International Conference on Human-Robot Interaction (pp. 317–324). IEEE/RSJ International Conference on Intelligent Robots and Systems Int’l Workshop on Human-Agent Interaction.

Hove, M. J., Iversen, J. R., Zhang, A., & Repp, B. H.

(2013) Synchronization with competing visual and auditory rhythms: bouncing ball meets metronome. Psychological Research, 77(4), 388–398.

Iio, T., Yoshikawa, Y., & Ishiguro, H.

(2016, October). Pre-scheduled turn-taking between robots to make conversation coherent. In Proceedings of the Fourth International Conference on Human Agent Interaction (pp. 19–25). ACM.

Iqbal, T., Moosaei, M., & Riek, L. D.

(2016) Tempo adaptation and anticipation methods for human-robot teams. RSS, Planning for HRI: Shared Autonomy and Collab. Robotics Work.

Jaffe, J., Beebe, B., Feldstein, S., Crown, C. L., Jasnow, M. D., Rochat, P., & Stern, D. N.

(2001) Rhythms of dialogue in infancy: Coordinated timing in development. Monographs of the society for research in child development, i–149.

Kawasaki, M., Yamada, Y., Ushiku, Y., Miyauchi, E., & Yamaguchi, Y.

(2013) Inter-brain synchronization during coordination of speech rhythm in human-to-human social interaction. Scientific reports, 31, 1692.

Kelso, J. S.

(1997) Dynamic patterns: The self-organization of brain and behavior. MIT press.

Kose-Bagci, H., Dautenhahn, K., & Nehaniv, C. L.

(2008, August). Emergent dynamics of turn-taking interaction in drumming games with a humanoid robot. In RO-MAN 2008-The 17th IEEE International Symposium on Robot and Human Interactive Communication (pp. 346–353). IEEE.

LaGasse, A. B., & Hardy, M. W.

(2013) Rhythm, movement, and autism: using rhythmic rehabilitation research as a model for autism. Frontiers in integrative neuroscience, 71, 19.

Lenggenhager, B., Tadi, T., Metzinger, T., & Blanke, O.

(2007) Video ergo sum: manipulating bodily self-consciousness. Science, 317(5841), 1096–1099.

Ma, K., & Hommel, B.

(2015) The role of agency for perceived ownership in the virtual hand illusion. Consciousness and cognition, 361, 277–288.

Mates, J.

(1994) A model of synchronization of motor acts to a stimulus sequence. Biological cybernetics, 70(5), 463–473.

McNeill, W. H.

(2009) Keeping Together in Time: Dance and Drill in Human History. Harvard Univ. Press.

Merker, B.

(1999) Synchronous chorusing and the origins of music. Musicae Scientiae, 3(1_suppl), 59–73.

Michalowski, M. P., Sabanovic, S., & Kozima, H.

(2007, March). A dancing robot for rhythmic social interaction. In 2nd ACM/IEEE International Conference on Human-Robot Interaction (HRI), 2007 (pp. 89–96). IEEE.

Michalowski, M. P., Simmons, R., & Kozima, H.

(2009, September). Rhythmic attention in child-robot dance play. In Robot and Human Interactive Communication, 2009. RO-MAN 2009. The 18th IEEE International Symposium on (pp. 816–821). IEEE.

Miles, L. K., Nind, L. K., & Macrae, C. N.

(2009) The rhythm of rapport: Interpersonal synchrony and social perception. Journal of experimental social psychology, 45(3), 585–589.

Milgram, S., Bickman, L., & Berkowitz, L.

(1969) Note on the drawing power of crowds of different size. Journal of personality and social psychology, 13(2), 79.

Milgram, S. & Gudehus, C.

(1978) Obedience to authority.

Mörtl, A., Lorenz, T., & Hirche, S.

(2014) Rhythm patterns interaction-synchronization behavior for human-robot joint action. PloS one, 9(4), e95195.

Néda, Z., Ravasz, E., Brechet, Y., Vicsek, T., & Barabási, A. L.

(2000) Tumultuous applause can transform itself into waves of synchronized clapping. Nature, 403(6772), 849–850.

Nishio, S., Watanabe, T., Ogawa, K., & Ishiguro, H.

(2012, October). Body ownership transfer to teleoperated android robot. In International Conference on Social Robotics (pp. 398–407). Springer, Berlin, Heidelberg.

Overy, K.

(2012) Making music in a group: synchronization and shared experience. Annals of the New York Academy of Sciences, 1252(1), 65–68.

Pikovsky, A., Rosenblum, M., & Kurths, J.

(2003) Synchronization: a universal concept in nonlinear sciences (Vol. 121). Cambridge university press.

Rau, P. L. P., Li, Y., & Liu, J.

(2013) Effects of a social robot’s autonomy and group orientation on human decision-making. Advances in Human-Computer Interaction 2013, 111.

Richardson, M. J., Marsh, K. L., Isenhower, R. W., Goodman, J. R., & Schmidt, R. C.

(2007) Rocking together: Dynamics of intentional and unintentional interpersonal coordination. Human movement science, 26(6), 867–891.

Richardson, M. J., Marsh, K. L., & Schmidt, R. C.

(2005) Effects of visual and verbal interaction on unintentional interpersonal coordination. Journal of Experimental Psychology: Human Perception and Performance, 31(1), 62.

Roederer, J. G.

(1984) The search for a survival value of music. Music Perception: An Interdisciplinary Journal, 1(3), 350–356.

Schloss, W. A.

(1985) On the automatic transcription of percussive music – From acoustic signals to high-level analysis (Doctoral dissertation, Stanford University).

Schmidt, R. C., Carello, C., & Turvey, M. T.

(1990) Phase transitions and critical fluctuations in the visual coordination of rhythmic movements between people. Journal of experimental psychology: human perception and performance, 16(2), 227.

Scothern, P. M. T.

(1992) The music-archaeology of the Palaeolithic within its cultural setting (Doctoral dissertation, University of Cambridge).

Shimada et al.

(2009) Rubber Hand Illusion under Delayed Visual Feedback, PLoS ONE, 4(7): e6185.

Slater, M., Pérez–Marcos, D., Ehrsson, H., & Sanchez–Vives, M. V.

(2008) Towards a digital body: the virtual arm illusion. Frontiers in human neuroscience, 21, 6.

Tsakiris, M., & Haggard, P.

(2005) The rubber hand illusion revisited: visuotactile integration and self-attribution. Journal of Experimental Psychology: Human Perception and Performance, 31(1), 80.

Valdesolo, P., Ouyang, J., & DeSteno, D.

(2010) The rhythm of joint action: Synchrony promotes cooperative ability. Journal of Experimental Social Psychology, 46(4), 693–695.

van Ulzen, N. R., Lamoth, C. J., Daffertshofer, A., Semin, G. R., & Beek, P. J.

(2008) Characteristics of instructed and uninstructed interpersonal coordination while walking side-by-side. Neuroscience letters, 432(2), 88–93.

Van Vugt, F. T., Ritter, J., Rollnik, J. D., & Altenmüller, E.

(2014) Music-supported motor training after stroke reveals no superiority of synchronization in group therapy. Frontiers in human neuroscience, 81, 315.

Wallach, M. A., Kogan, N., & Bem, D. J.

(1962) Group influence on individual risk taking. ETS Research Report Series 1962(1).

Wilder, D. A.

(1977) Perception of groups, size of opposition, and social influence. Journal of Experimental Social Psychology, 13(3), 253–268.