Cover not available
Article published In:
Interaction Studies
Vol. 20:2 (2019) ► pp.205–233

Socio-emotional development in high functioning children with Autism Spectrum Disorders using a humanoid robot

| University of Minho
| University of Minho
| University of Minho
| University of Minho
| University of Hertfordshire
| University of Minho

The use of robots had already been proven to encourage the promotion of social interaction and skills lacking in children with Autism Spectrum Disorders (ASD), who typically have difficulties in recognizing facial expressions and emotions. The main goal of this research is to study the influence of a humanoid robot to develop socio-emotional skills in children with ASD. The children’s performance in game scenarios aiming to develop facial expressions recognition skills is presented. Along the sessions, children who performed the game scenarios with the robot and the experimenter had a significantly better performance than the children who performed the game scenarios without the robot. The main conclusions of this research support that a humanoid robot is a useful tool to develop socio-emotional skills in the intervention of children with ASD, due to the engagement and positive learning outcome observed.

Keywords: human-robot interaction, Autism Spectrum Disorders, emotion recognition
Article outline
  • 1.Introduction
  • 2.Background
    • 2.1Facial expressions displayed by robots
    • 2.2Emotion recognition difficulty
    • 2.3Research questions
  • 3.Methods
    • 3.1Ethics statement
    • 3.2Participants
    • 3.3Procedures
      • Performance task
      • Practice phase
    • 3.4Focus groups
    • 3.5The robot
    • 3.6Robot’s input and processing
    • 3.7Experimental setup
    • 3.8Evaluation tools
  • 4.Results
  • 4.1Comparison of the pre- and post-test data between G1, G2, and G3
  • 5.Discussion of the results
    • 5.1Summary of hypotheses and implications
  • 6.Conclusions and future work
  • Acknowledgements
  • References
Published online: 7 October 2019
DOI logo
https://doi.org/10.1075/is.15003.cos
References
Association, A.
(2013) Diagnostic and Statistical Manual of Mental Disorders, 5th Edition: DSM 5. book-pointUS. DOI logoGoogle Scholar
Bakeman, R. and Gottman, J.
(1997) Observing interaction: An introduction to sequential analysis. Cambridge Univ Pr.. DOI logoGoogle Scholar
Baron-Cohen, S.
(1991) The development of a theory of mind in autism: deviance and delay? Psychiatric Clinics of North America. DOI logoGoogle Scholar
Boucenna, S., Narzisi, A., Tilmont, E., Muratori, F., Pioggia, G., Cohen, D., and Chetouani, M.
(2014) Interactive technologies for autistic children: A review. Cognitive Computation, pages 1–19.Google Scholar
Cabibihan, J.-J., Javed, H., Jr., M. H. A., and Aljunied, S. M.
(2013) Why robots? a survey on the roles and benefits of social robots in the therapy of children with autism. CoRR, abs/1311.0352.Google Scholar
Celani, G., Battacchi, M., and Arcidiacono, L.
(1999) The understanding of the emotional meaning of facial expressions in people with autism. Journal of Autism and Developmental Disorders, 29(1):57–66. DOI logoGoogle Scholar
Charman, T. and Stone, W. L.
(2006) Social and communication development in autism spectrum disorders: Early identification, diagnosis, and intervention. Guilford Press.Google Scholar
Clark, T. F., Winkielman, P., and Mcintosh, D. N.
(2008) Autism and the extraction of emotion from briefly presented facial expressions: Stumbling at the first step of empathy. Emotion, 8(6):803. DOI logoGoogle Scholar
Costa, S., Soares, F., Pereira, A. P., Santos, C., and Hiolle, A.
(2014a) Building a game scenario to encourage children with autism to recognize and label emotions using a humanoid robot. in Robot and Human Interactive Communication, 2014 RO-MAN: The 23rd IEEE International Symposium on, pages 820–825. IEEE.Google Scholar
Costa, S., Soares, F., and Santos, C.
(2013) Facial expressions and gestures to convey emotions with a hu-manoid robot. In Herrmann, G., Pearson, M., Lenz, A., Bremner, P., Spiers, A., and Leonards, U., editors, Social Robotics, volume 8239 of Lecture Notes in Computer Science, pages 542–551. Springer International Publishing.Google Scholar
Costa, S., Soares, F. O., Pereira, A. P., Santos, C., and Hiolle, A.
(2014b) A pilot study using imitation and storytelling scenarios as activities for labelling emotions by children with autism using a humanoid robot. In Proceeding of The 4th International Conference on Development and Learning and on Epigenetic Robotics (IEEE ICDL-EPIROB 2014). DOI logoGoogle Scholar
Dautenhahn, K.
(1999) Robots as social actors: Aurora and the case of autism. In Proc. CT99, The Third International Cognitive Technology Conference, August, San Francisco, volume 3591, page 374.Google Scholar
Dautenhahn, K. and Werry, I.
(2004) Towards interactive robots in autism therapy: Background, motivation and challenges. Pragmatics & Cognition, 12(1):1–35. DOI logoGoogle Scholar
Diehl, J. J., Schmitt, L. M., Villano, M., and Crowell, C. R.
(2012) The clinical use of robots for individuals with autism spectrum disorders: A critical review. Research in autism spectrum disorders, 6(1):249–262. DOI logoGoogle Scholar
Ekman, P. and Rosenberg, E. L.
(1998) What the face reveals: Basic and applied studies of spontaneous expression using the Facial Action Coding System (FACS). Oxford University Press, USA.Google Scholar
Filipe, C.
(2012) Autismo: conceitos, mitos e preconceitos.Google Scholar
Giullian, N., Ricks, D., Atherton, A., Colton, M., Goodrich, M., and Brinton, B.
(2010) Detailed requirements for robots in autism therapy. In Systems Man and Cybernetics (SMC), 2010 IEEE International Conference on, pages 2595–2602. IEEE. DOI logoGoogle Scholar
Hanson, D., Baurmann, S., Riccio, T., Margolin, R., Dockins, T., Tavares, M., and Carpenter, K.
(2009) Zeno: a cognitive character. In AI Magazine, and special Proc. of AAAI National Conference, Chicago.Google Scholar
Hobson, R. P.
(1986) The autistic child’s appraisal of expressions of emotion: A further study. Journal of Child Psychology and Psychiatry, 27(5):671–680. DOI logoGoogle Scholar
Huskens, B., Verschuur, R., Gillesen, J., Didden, R., and Barakova, E.
(2013) Promoting question-asking in school-aged children with autism spectrum disorders: Effectiveness of a robot intervention compared to a human-trainer intervention. Developmental neurorehabilitation, 16(5):345–356. DOI logoGoogle Scholar
Johnson, C. P., Myers, S. M., et al.
(2007) Identification and evaluation of children with autism spectrum disorders. Pediatrics, 120(5):1183–1215. DOI logoGoogle Scholar
Kanade, T., Cohn, J. F., and Tian, Y.
(2000) Comprehensive database for facial expression analysis. In Automatic Face and Gesture Recognition, 2000. Proceedings. Fourth IEEE International Conference on, pages 46–53. IEEE.Google Scholar
Lee, J., Takehashi, H., Nagai, C., Obinata, G., and Stefanov, D.
(2012) Which robot features can stimulate better responses from children with autism in robotassisted therapy? Int. J. Advanced Robotic Systems, 9(72).Google Scholar
Lee, K. M., Jung, Y., Kim, J., and Kim, S. R.
(2006) Are physically embodied social agents better than disembodied social agents?: The effects of physical embodiment, tactile interaction, and people’s loneliness in human-robot interaction. International Journal of Human-Computer Studies, 64(10):962–973. DOI logoGoogle Scholar
Lima, C. B. D.
(2012) Perturbações do espectro do autismo: manual prático de intervençao. Lidel.Google Scholar
Loveland, K. A., Tunali-Kotoski, B., Chen, Y. R., Ortegon, J., Pearson, D. A., Brelsford, K. A., and Gibbs, M. C.
(1997) Emotion recognition in autism: Verbal and nonverbal information. Development and Psychopathology, null1:579–593. DOI logoGoogle Scholar
Lucey, P., Cohn, J. F., Kanade, T., Saragih, J., Ambadar, Z., and Matthews, I.
(2010) The extended cohn-kanade dataset (ck+): A complete dataset for action unit and emotion-specified expression. In Computer Vision and Pattern Recognition Workshops (CVPRW), 2010 IEEE Computer Society Conference on, pages 94–101. IEEE.Google Scholar
Mazzei, D., Lazzeri, N., Billeci, L., Igliozzi, R., Mancini, A., Ahluwalia, A., Muratori, F., and De Rossi, D.
(2011) Development and evaluation of a social robot platform for therapy in autism. In Engineering in Medicine and Biology Society, EMBC, 2011 Annual International Conference of the IEEE, pages 4515–4518. IEEE. DOI logoGoogle Scholar
Mazzei, D., Lazzeri, N., Hanson, D., and De Rossi, D.
(2012) Hefes: an hybrid engine for facial expressions synthesis to control human-like androids and avatars. In BIOROB 2012 proceedings.Google Scholar
Mesibov, G. B. and Howley, M.
(2003) Accessing the curriculum for pupils with autistic spectrum disorders: Using the TEACCH programme to help inclusion. David Fulton Publishers.Google Scholar
Noldus, L.
(1991) The observer: A software system for collection and analysis of observational data. Behavior Research Methods, Instruments, & Computers, 23(3):415–429. DOI logoGoogle Scholar
Palmen, A., Didden, R., and Lang, R.
(2012) A systematic review of behavioral intervention research on adaptive skill building in high-functioning young adults with autism spectrum disorder. Research in Autism Spectrum Disorders, 6(2):602–617. DOI logoGoogle Scholar
Pease, B. and Pease, A.
(2008) The definitive book of body language. Random House LLC.Google Scholar
Ramdoss, S., Lang, R., Mulloy, A., Franco, J., OReilly, M., Didden, R., and Lancioni, G.
(2011) Use of computer-based interventions to teach communication skills to children with autism spectrum disorders: A systematic review. Journal of Behavioral Education, 20(1):55–76. DOI logoGoogle Scholar
Ramdoss, S., Machalicek, W., Rispoli, M., Mulloy, A., Lang, R., and OReilly, M.
(2012) Computer-based interventions to improve social and emotional skills in individuals with autism spectrum disorders: A systematic review. Developmental neurorehabilitation, 15(2):119–135. DOI logoGoogle Scholar
Ricks, D. J. and Colton, M. B.
(2010) Trends and considerations in robot-assisted autism therapy. In Robotics and Automation (ICRA), 2010 IEEE International Conference on, pages 4354–4359. IEEE. DOI logoGoogle Scholar
Russell, J. A.
(1980) A circumplex model of affect. Journal of personality and social psychology, 39(6):1161. DOI logoGoogle Scholar
Saldien, J., Goris, K., Vanderborght, B., Vanderfaeillie, J., and Lefeber, D.
(2010) Expressing emotions with the social robot probo. International Journal of Social Robotics, 2(4):377–389. DOI logoGoogle Scholar
Scassellati, B., Admoni, H., and Mataric, M.
(2012) Robots for use in autism research. Annual Review of Biomedical Engineering, 141:275–294. DOI logoGoogle Scholar
Werry, I., Dautenhahn, K., Ogden, B., and Harwin, W.
(2001) Can social interaction skills be taught by a social agent? the role of a robotic mediator in autism therapy. In Cognitive technology: instruments of mind, pages 57–74. Springer. DOI logoGoogle Scholar
Zwaigenbaum, L., Bryson, S., Rogers, T., Roberts, W., Brian, J., and Szatmari, P.
(2005) Behavioral manifestations of autism in the first year of life. International Journal of Developmental Neuroscience, 23(2):143–152. DOI logoGoogle Scholar
Cited by

Cited by 14 other publications

Abdelmohsen, Maha & Yasmine Arafa
2021. Virtual Social Robot Enhances the Social Skills of Children with HFA. In Social Robotics [Lecture Notes in Computer Science, 13086], ► pp. 497 ff. DOI logo
Alghamdi, Maha, Noura Alhakbani & Abeer Al-Nafjan
2023. Assessing the Potential of Robotics Technology for Enhancing Educational for Children with Autism Spectrum Disorder. Behavioral Sciences 13:7 ► pp. 598 ff. DOI logo
Alimardani, Maryam, Lisa Neve & Anouk Verkaart
2021. Storytelling Robots for Training of Emotion Recognition in Children with Autism; Opinions from Experts. In HCI International 2021 - Late Breaking Posters [Communications in Computer and Information Science, 1498], ► pp. 223 ff. DOI logo
Chen, Xieling, Di Zou, Lucas Kohnke, Haoran Xie, Gary Cheng & Mohammed Saqr
2021. Affective states in digital game-based learning: Thematic evolution and social network analysis. PLOS ONE 16:7 ► pp. e0255184 ff. DOI logo
Damianidou, Despoina, Ami Eidels & Michael Arthur-Kelly
2020. The Use of Robots in Social Communications and Interactions for Individuals with ASD: a Systematic Review. Advances in Neurodevelopmental Disorders 4:4 ► pp. 357 ff. DOI logo
Holeva, Vasiliki, V. A. Nikopoulou, C. Lytridis, C. Bazinas, P. Kechayas, G. Sidiropoulos, M. Papadopoulou, M. D. Kerasidou, C. Karatsioras, N. Geronikola, G. A. Papakostas, V. G. Kaburlasos & A. Evangeliou
2024. Effectiveness of a Robot-Assisted Psychological Intervention for Children with Autism Spectrum Disorder. Journal of Autism and Developmental Disorders 54:2 ► pp. 577 ff. DOI logo
Kewalramani, Sarika, Kelly-Ann Allen, Erin Leif & Andrea Ng
2023. A Scoping Review of the Use of Robotics Technologies for Supporting Social-Emotional Learning in Children with Autism. Journal of Autism and Developmental Disorders DOI logo
Lecciso, Flavia, Annalisa Levante, Rosa Angela Fabio, Tindara Caprì, Marco Leo, Pierluigi Carcagnì, Cosimo Distante, Pier Luigi Mazzeo, Paolo Spagnolo & Serena Petrocchi
2021. Emotional Expression in Children With ASD: A Pre-Study on a Two-Group Pre-Post-Test Design Comparing Robot-Based and Computer-Based Training. Frontiers in Psychology 12 DOI logo
Neerincx, Anouk, Yanzhe Li, Kelvin van de Sande, Frank Broz, Mark Neerincx & Maartje de Graaf
2023. 2023 32nd IEEE International Conference on Robot and Human Interactive Communication (RO-MAN), ► pp. 1202 ff. DOI logo
Silva, Vinicius, Filomena Soares, João Sena Esteves & Ana Paula Pereira
2018. Building a Hybrid Approach for a Game Scenario Using a Tangible Interface in Human Robot Interaction. In Serious Games [Lecture Notes in Computer Science, 11243], ► pp. 241 ff. DOI logo
Silva, Vinícius, Ana Paula Pereira, Filomena Soares, Celina P. Leão, Andrea Jurdi, João Sena Esteves & Juliana Hertzberg
2023. Social Stories for Promoting Social Communication with Children with Autism Spectrum Disorder Using a Humanoid Robot: Step-by-Step Study. Technology, Knowledge and Learning DOI logo
Silva, Vinícius, Filomena Soares, João Sena Esteves, Cristina P. Santos & Ana Paula Pereira
2021. Fostering Emotion Recognition in Children with Autism Spectrum Disorder. Multimodal Technologies and Interaction 5:10 ► pp. 57 ff. DOI logo
Silva, Vinícius, Filomena Soares, Celina P. Leão, João Sena Esteves & Gianni Vercelli
2021. Skeleton Driven Action Recognition Using an Image-Based Spatial-Temporal Representation and Convolution Neural Network. Sensors 21:13 ► pp. 4342 ff. DOI logo
Soleiman, Pegah, Hadi Moradi, Bijan Mehralizadeh, Hamed Ameri, Rosa I. Arriaga, Hamid Reza Pouretemad, Negin Baghbanzadeh & Leila Kashani Vahid
2023. Fully robotic social environment for teaching and practicing affective interaction: Case of teaching emotion recognition skills to children with autism spectrum disorder, a pilot study. Frontiers in Robotics and AI 10 DOI logo

This list is based on CrossRef data as of 31 march 2024. Please note that it may not be complete. Sources presented here have been supplied by the respective publishers. Any errors therein should be reported to them.