Article published In:
Psychological Benchmarks of Human–Robot InteractionEdited by Peter H. Kahn, Jr. and Karl F. MacDorman
[Interaction Studies 8:3] 2007
► pp. 423–439
Edited by Peter H. Kahn, Jr. and Karl F. MacDorman
[Interaction Studies 8:3] 2007
► pp. 423–439
Benchmarks for evaluating socially assistive robotics
Socially assistive robotics (SAR) is a growing area of research. Evaluating SAR systems presents novel challenges. Using a robot for a socially assistive task can have various benefits and ethical implications. Many questions are important to understanding whether a robot is effective for a given application domain. This paper describes several benchmarks for evaluating SAR systems. There exist numerous methods for evaluating the many factors involved in a robot’s design. Benchmarks from psychology, anthropology, medicine, and human–robot interaction are proposed as measures of success in evaluating a given SAR system and its impact on the user and broader population.
Cited by (89)
Cited by 89 other publications
Cruz-Sandoval, Dagoberto, Monica Tentori & Jesus Favela
2025. A Framework to Design Engaging Interactions in Socially Assistive Robots to Mitigate Dementia-Related Symptoms. ACM Transactions on Human-Robot Interaction 14:1 ► pp. 1 ff. 
Liu, Tangyou, Jiaole Wang, Shing Wong, Andrew Razjigaev, Susann Beier, Shuhua Peng, Thanh Nho Do, Shuang Song, Dewei Chu, Chun Hui Wang, Nigel H. Lovell & Liao Wu
2024. A Review on the Form and Complexity of Human–Robot Interaction in the Evolution of Autonomous Surgery. Advanced Intelligent Systems 6:11
Fazlollahi, Farimah & Katherine J. Kuchenbecker
2023. Haptify: A Measurement-Based Benchmarking System for Grounded Force-Feedback Devices. IEEE Transactions on Robotics 39:2 ► pp. 1622 ff.
Hermann, Julia, Selcan Oktay, Artur Lisetschko & Aysegül Dogangün
2023. Proceedings of the 35th Australian Computer-Human Interaction Conference, ► pp. 221 ff.
Meister, Martin
2023. An Institutional Field as a Heterogeneous Innovation Context: The Example of Social Robotics. In Berlin Keys to the Sociology of Technology, ► pp. 261 ff.
Adlakha, Deepansha & Dolly Sharma
2022. Human‐Robot Interaction: An Artificial Cognition‐Based Study for Criminal Investigations. In Digital Forensics and Internet of Things, ► pp. 163 ff.
Devaraj, Uthirakumar & Balamurugan Elumalai
2022. Robot Therapy. In Handbook of Research on Lifestyle Sustainability and Management Solutions Using AI, Big Data Analytics, and Visualization [Advances in Computational Intelligence and Robotics, ], ► pp. 137 ff.
Elbeleidy, Saad, Terran Mott & Tom Williams
2022. 2022 17th ACM/IEEE International Conference on Human-Robot Interaction (HRI), ► pp. 577 ff.
Lima, Maria R., Maitreyee Wairagkar, Manish Gupta, Ferdinando Rodriguez y Baena, Payam Barnaghi, David J. Sharp & Ravi Vaidyanathan
2022. Conversational Affective Social Robots for Ageing and Dementia Support. IEEE Transactions on Cognitive and Developmental Systems 14:4 ► pp. 1378 ff.
Schulze, Hartmut, Simone Kauffeld & Alexandra Tanner
2022. Soziale Roboter auf dem Weg in die Praxis. Gruppe. Interaktion. Organisation. Zeitschrift für Angewandte Organisationspsychologie (GIO) 53:3 ► pp. 267 ff.
Sengupta, Rounak & Vivek Kant
2022. Holistic Asset Performance Management Systems: Speculative design for Industrial APM systems and human work experience in an Industry 5.0 setting. INCOSE International Symposium 32:S1 ► pp. 182 ff.
Ahmad, Sayed Fayaz, Mohd. Khairil Rahmat, Muhammad Shujaat Mubarik, Muhammad Mansoor Alam & Syed Irfan Hyder
2021. Artificial Intelligence and Its Role in Education. Sustainability 13:22 ► pp. 12902 ff.
Franke, Annette, Elmar Nass, Anna-Kathleen Piereth, Annabel Zettl & Christian Heidl
2021. Implementation of Assistive Technologies and Robotics in Long-Term Care Facilities: A Three-Stage Assessment Based on Acceptance, Ethics, and Emotions. Frontiers in Psychology 12
Han, Isla Xi, Forrest Meggers & Stefana Parascho
2021. Bridging the collectives: A review of collective human–robot construction. International Journal of Architectural Computing 19:4 ► pp. 512 ff.
Onnasch, Linda & Eileen Roesler
2021. A Taxonomy to Structure and Analyze Human–Robot Interaction. International Journal of Social Robotics 13:4 ► pp. 833 ff.
Thoduka, Santosh & Nico Hochgeschwender
2021. Benchmarking Robots by Inducing Failures in Competition Scenarios. In Digital Human Modeling and Applications in Health, Safety, Ergonomics and Risk Management. AI, Product and Service [Lecture Notes in Computer Science, 12778], ► pp. 263 ff.
Vasylkiv, Yurii, Ricardo Ragel, Javier Ponce-Chulani, Luis Merino, Eleanor Sandry, Heike Brock, Keisuke Nakamura, Irani Pourang & Randy Gomez
2021. Design and Development of a Teleoperation System for Affective Tabletop Robot Haru. In Social Robotics [Lecture Notes in Computer Science, 13086], ► pp. 564 ff.
Baraka, Kim, Patrícia Alves-Oliveira & Tiago Ribeiro
2020. An Extended Framework for Characterizing Social Robots. In Human-Robot Interaction [Springer Series on Bio- and Neurosystems, 12], ► pp. 21 ff.
Bonarini, Andrea
2020. Communication in Human-Robot Interaction. Current Robotics Reports 1:4 ► pp. 279 ff.
Brenčič, Neja Samar, Marius Dragoi, Irina Mocanu & Tomasz Winiarski
2020. Intuitive and Intelligent Solutions for Elderly Care. In Digital Health in Focus of Predictive, Preventive and Personalised Medicine [Advances in Predictive, Preventive and Personalised Medicine, 12], ► pp. 101 ff.
Jahan, Farha, Weiqing Sun, Quamar Niyaz & Mansoor Alam
2020. Security Modeling of Autonomous Systems. ACM Computing Surveys 52:5 ► pp. 1 ff.
Mayima, Amandine, Aurelie Clodic & Rachid Alami
2020. 2020 29th IEEE International Conference on Robot and Human Interactive Communication (RO-MAN), ► pp. 291 ff.
Mois, George & Jenay M. Beer
2020. Robotics to support aging in place. In Living with Robots, ► pp. 49 ff.
Mois, George & Jenay M. Beer
2020. The Role of Healthcare Robotics in Providing Support to Older Adults: a Socio-ecological Perspective. Current Geriatrics Reports 9:2 ► pp. 82 ff.
Rahaman, Ashiqur & Byungki Kim
2020. 2020 20th International Conference on Control, Automation and Systems (ICCAS), ► pp. 1120 ff.
Rossi, Silvia, Alessandra Rossi & Kerstin Dautenhahn
2020. The Secret Life of Robots: Perspectives and Challenges for Robot’s Behaviours During Non-interactive Tasks. International Journal of Social Robotics 12:6 ► pp. 1265 ff.
Werner, Franz
2020. A Survey on Current Practices in User Evaluation of Companion Robots. In Human-Robot Interaction [Springer Series on Bio- and Neurosystems, 12], ► pp. 65 ff.
Čaić, Martina, João Avelino, Dominik Mahr, Gaby Odekerken-Schröder & Alexandre Bernardino
2020. Robotic Versus Human Coaches for Active Aging: An Automated Social Presence Perspective. International Journal of Social Robotics 12:4 ► pp. 867 ff.
Aguiar Noury, Gabriel, Hannah Bradwell, Serge Thill & Ray Jones
2019. User-defined challenges and desiderata for robotics and autonomous systems in health and social care settings. Advanced Robotics 33:7-8 ► pp. 309 ff.
Ammon, Sabine & Martin Meister
2019. Roboter, humanoide. In Mensch-Maschine-Interaktion, ► pp. 297 ff.
Fiederer, Lukas D. J., Martin Völker, Robin T. Schirrmeister, Wolfram Burgard, Joschka Boedecker & Tonio Ball
2019. Hybrid Brain-Computer-Interfacing for Human-Compliant Robots: Inferring Continuous Subjective Ratings With Deep Regression. Frontiers in Neurorobotics 13
Fortuna, Karen L., Maria Venegas, Emre Umucu, George Mois, Robert Walker & Jessica M. Brooks
2019. The Future of Peer Support in Digital Psychiatry: Promise, Progress, and Opportunities. Current Treatment Options in Psychiatry 6:3 ► pp. 221 ff.
Park, Chung Hyuk, Hifza Javed & Myounghoon Jeon
2019. Consensus-Based Human-Agent Interaction Model for Emotion Regulation in ASD. In HCI International 2019 - Posters [Communications in Computer and Information Science, 1032], ► pp. 295 ff.
Welfare, Katherine S., Matthew R. Hallowell, Julie A. Shah & Laurel D. Riek
2019. 2019 14th ACM/IEEE International Conference on Human-Robot Interaction (HRI), ► pp. 75 ff.
Čaić, Martina, Dominik Mahr & Gaby Oderkerken-Schröder
2019. Value of social robots in services: social cognition perspective. Journal of Services Marketing 33:4 ► pp. 463 ff.
Ajoudani, Arash, Andrea Maria Zanchettin, Serena Ivaldi, Alin Albu-Schäffer, Kazuhiro Kosuge & Oussama Khatib
2018. Progress and prospects of the human–robot collaboration. Autonomous Robots 42:5 ► pp. 957 ff.
Chang, Rebecca Cherng-Shiow, Hsi-Peng Lu & Peishan Yang
2018. Stereotypes or golden rules? Exploring likable voice traits of social robots as active aging companions for tech-savvy baby boomers in Taiwan. Computers in Human Behavior 84 ► pp. 194 ff.
Schulz, Trenton, Jo Herstad & Harald Holone
2018. Privacy at Home: An Inquiry into Sensors and Robots for the Stay at Home Elderly. In Human Aspects of IT for the Aged Population. Applications in Health, Assistance, and Entertainment [Lecture Notes in Computer Science, 10927], ► pp. 377 ff.
Čaić, Martina, Gaby Odekerken-Schröder & Dominik Mahr
2018. Service robots: value co-creation and co-destruction in elderly care networks. Journal of Service Management 29:2 ► pp. 178 ff.
Aly, Amir, Sascha Griffiths & Francesca Stramandinoli
2017. Metrics and benchmarks in human-robot interaction: Recent advances in cognitive robotics. Cognitive Systems Research 43 ► pp. 313 ff.
Bodine, Cathy, Levin Sliker, Michael Marquez, Cecilia Clark, Brian Burne & Jim Sandstrum
2017. 8. Social Assistive Robots for Children with Complex Disabilities. In Robotic Assistive Technologies [Rehabilitation Science in Practice Series, ], ► pp. 261 ff.
JEREMY, (ZHENG) LI
2017. NEW TECHNOLOGY IN COMPUTER AIDED DESIGN, COMPUTER AIDED MANUFACTURING, COMPUTER AIDED ENGINEERING ANALYTIC METHODOLOGY, AUTOMATED AND HIGH SPEED MECHANICAL SYSTEM. i-manager's Journal on Computer Science 5:3 ► pp. 6 ff.
Kipp, Andreas & Sebastian Schneider
2017. Applied Social Robotics—Building Interactive Robots with LEGO Mindstorms. In Robotics in Education [Advances in Intelligent Systems and Computing, 457], ► pp. 29 ff.
Salehuddin, Muhammad, Suprijanto & Augie Widyotriatmo
2017. 2017 International Conference on Smart Cities, Automation & Intelligent Computing Systems (ICON-SONICS), ► pp. 147 ff.
Schulz, Trenton & Jo Herstad
2017. [Electronic Workshops in Computing, ],
Shukla, Jainendra, Miguel Barreda-Angeles, Joan Oliver & Domenec Puig
2017. 2017 26th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN), ► pp. 62 ff.
Subin, E. K., Ashik Hameed & A. P. Sudheer
2017. Proceedings of the Advances in Robotics, ► pp. 1 ff.
Coeckelbergh, Mark, Cristina Pop, Ramona Simut, Andreea Peca, Sebastian Pintea, Daniel David & Bram Vanderborght
2016. A Survey of Expectations About the Role of Robots in Robot-Assisted Therapy for Children with ASD: Ethical Acceptability, Trust, Sociability, Appearance, and Attachment. Science and Engineering Ethics 22:1 ► pp. 47 ff.
Compagna, Diego, Manuela Marquardt & Ivo Boblan
2016. Introducing a Methodological Approach to Evaluate HRI from a Genuine Sociological Point of View. In Cultural Robotics [Lecture Notes in Computer Science, 9549], ► pp. 55 ff.
Corti, Kevin & Alex Gillespie
2016. Co-constructing intersubjectivity with artificial conversational agents: People are more likely to initiate repairs of misunderstandings with agents represented as human. Computers in Human Behavior 58 ► pp. 431 ff.
Esposito, Anna & Lakhmi C. Jain
2016. Modeling Social Signals and Contexts in Robotic Socially Believable Behaving Systems. In Toward Robotic Socially Believable Behaving Systems - Volume II [Intelligent Systems Reference Library, 106], ► pp. 5 ff.
Huber, Andreas, Astrid Weiss & Marjo Rauhala
2016. 2016 11th ACM/IEEE International Conference on Human-Robot Interaction (HRI), ► pp. 367 ff.
Insaurralde, Carlos C. & Thilo Kaupisch
2016. 2016 IEEE Aerospace Conference, ► pp. 1 ff.
Insaurralde, Carlos C. & Thilo Kaupisch
2018. 2018 IEEE Aerospace Conference, ► pp. 1 ff.
Le, Hoang H., Martin J. Loomes & Rui C.V. Loureiro
2016. 2016 12th International Conference on Intelligent Environments (IE), ► pp. 62 ff.
Matsuzaki, Hironori & Gesa Lindemann
2016. The autonomy-safety-paradox of service robotics in Europe and Japan: a comparative analysis. AI & SOCIETY 31:4 ► pp. 501 ff.
Angulo, Cecilio, Sammy Pfeiffer, Ricardo Tellez, Guillem Alenyà, Stefano Chessa, Juan Antonio Alvarez García & Paolo Barsocchi
2015. Evaluating the use of robots to enlarge AAL services1. Journal of Ambient Intelligence and Smart Environments 7:3 ► pp. 301 ff.
Beer, Jenay M., Arthur D. Fisk & Wendy A. Rogers
2015. Commanding Home Robots. Proceedings of the Human Factors and Ergonomics Society Annual Meeting 59:1 ► pp. 70 ff.
Compagna, Diego & Ivo Boblan
2015. Case-Sensitive Methods for Evaluating HRI from a Sociological Point of View. In Social Robotics [Lecture Notes in Computer Science, 9388], ► pp. 155 ff.
Esposito, Anna, Antonietta M. Esposito & Carl Vogel
2015. Needs and challenges in human computer interaction for processing social emotional information. Pattern Recognition Letters 66 ► pp. 41 ff.
Hersh, Marion
2015. Overcoming Barriers and Increasing Independence – Service Robots for Elderly and Disabled People. International Journal of Advanced Robotic Systems 12:8
Li, Zheng (Jeremy)
2015. Robotic System Design and Development for Medical Treatment, Surgery, and Automated Production Applications. In Handbook of Research on Advancements in Robotics and Mechatronics [Advances in Computational Intelligence and Robotics, ], ► pp. 851 ff.
Rabbitt, Sarah M., Alan E. Kazdin & Brian Scassellati
2015. Integrating socially assistive robotics into mental healthcare interventions: Applications and recommendations for expanded use. Clinical Psychology Review 35 ► pp. 35 ff.
Trautman, Pete, Jeremy Ma, Richard M. Murray & Andreas Krause
2015. Robot navigation in dense human crowds: Statistical models and experimental studies of human–robot cooperation. The International Journal of Robotics Research 34:3 ► pp. 335 ff.
Zheng Li, Jeremy
2015. Magnetic Sealing System. In CAD, 3D Modeling, Engineering Analysis, and Prototype Experimentation, ► pp. 129 ff.
Fridin, Marina
2014. Kindergarten social assistive robot: First meeting and ethical issues. Computers in Human Behavior 30 ► pp. 262 ff.
McColl, D., Wing-Yue Geoffrey Louie & G. Nejat
2013. Brian 2.1: A socially assistive robot for the elderly and cognitively impaired. IEEE Robotics & Automation Magazine 20:1 ► pp. 74 ff.
Tan, Yeow Kee, Alvin Wong, Anthony Wong, Tran Anh Dung, Adrian Tay, Dilip Limbu Kumar, Tran Huy Dat, Weng Zheng Ng, Rui Yan & Benedict Tay
2013. Evaluation of the Pet Robot CuDDler Using Godspeed Questionnaire. In Inclusive Society: Health and Wellbeing in the Community, and Care at Home [Lecture Notes in Computer Science, 7910], ► pp. 102 ff.
Vazquez-Santacruz, Eduardo & Mariano Gamboa-Zuniga
2013. 2013 10th International Conference on Electrical Engineering, Computing Science and Automatic Control (CCE), ► pp. 163 ff.
Caine, Kelly, Selma Šabanovic & Mary Carter
2012. Proceedings of the seventh annual ACM/IEEE international conference on Human-Robot Interaction, ► pp. 343 ff.
Groten, Raphaela, Daniela Feth, Angelika Peer & Martin Buss
2012. Psychological Experiments in Haptic Collaboration Research. In Immersive Multimodal Interactive Presence [Springer Series on Touch and Haptic Systems, ], ► pp. 65 ff.
Feil-Seifer, David & Maja Mataric
2011. Socially Assistive Robotics. IEEE Robotics & Automation Magazine 18:1 ► pp. 24 ff.
Feth, Daniela
2011. Haptic Human–Robot Collaboration: Comparison of Robot Partner Implementations in Terms of Human-Likeness and Task Performance. Presence: Teleoperators and Virtual Environments 20:2 ► pp. 173 ff.
Gadde, Prathik, Hadi Kharrazi, Himalaya Patel & Karl F. MacDorman
2011. Toward Monitoring and Increasing Exercise Adherence in Older Adults by Robotic Intervention: A Proof of Concept Study. Journal of Robotics 2011 ► pp. 1 ff.
Mitchell, Wade J., Chin-Chang Ho, Himalaya Patel & Karl F. MacDorman
2011. Does social desirability bias favor humans? Explicit–implicit evaluations of synthesized speech support a new HCI model of impression management. Computers in Human Behavior 27:1 ► pp. 402 ff.
Nikolopoulos, Chris, Deitra Kuester, Mark Sheehan & Sneha Dhanya
2011. Investigation on Requirements of Robotic Platforms to Teach Social Skills to Individuals with Autism. In Human-Robot Personal Relationships [Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering, 59], ► pp. 65 ff.
van der Zant, Tijn & Luca Iocchi
2011. RoboCup
Home: Adaptive Benchmarking of Robot Bodies and Minds. In Social Robotics [Lecture Notes in Computer Science, 7072], ► pp. 214 ff.
Home: Adaptive Benchmarking of Robot Bodies and Minds. In Social Robotics [Lecture Notes in Computer Science, 7072], ► pp. 214 ff. Yamaji, Yuto, Taisuke Miyake, Yuta Yoshiike, P. Ravindra S. De Silva & Michio Okada
2011. STB: Child-Dependent Sociable Trash Box. International Journal of Social Robotics 3:4 ► pp. 359 ff.
Nikolopoulos, Chris, Deitra Kuester, Mark Sheehan, Sneha Dhanya, William Herring, Andrew Becker & Lauralyn Bogart
2010. Socially Assistive Robots and Autism. Solid State Phenomena 166-167 ► pp. 315 ff.
Broadbent, E., R. Stafford & B. MacDonald
2009. Acceptance of Healthcare Robots for the Older Population: Review and Future Directions. International Journal of Social Robotics 1:4 ► pp. 319 ff.
Broadbent, Elizabeth, Rie Tamagawa, Ngaire Kerse, Brett Knock, Anna Patience & Bruce MacDonald
2009. RO-MAN 2009 - The 18th IEEE International Symposium on Robot and Human Interactive Communication, ► pp. 645 ff.
Cerulo, Karen A.
2009. Nonhumans in Social Interaction. Annual Review of Sociology 35:1 ► pp. 531 ff.
Cerulo, Karen A.
2011. Social Interaction: Do Non‐humans Count?. Sociology Compass 5:9 ► pp. 775 ff.
MacDorman, Karl F., Sandosh K. Vasudevan & Chin-Chang Ho
2009. Does Japan really have robot mania? Comparing attitudes by implicit and explicit measures. AI & SOCIETY 23:4 ► pp. 485 ff.
Feil-Seifer, David & Maja J. Mataric
2008. RO-MAN 2008 - The 17th IEEE International Symposium on Robot and Human Interactive Communication, ► pp. 328 ff.
Feil-Seifer, David & Maja J. Matarić
2009. Human Robot Interaction. In Encyclopedia of Complexity and Systems Science, ► pp. 4643 ff.
Feil-Seifer, David & Maja J. Matarić
2009. Toward Socially Assistive Robotics for Augmenting Interventions for Children with Autism Spectrum Disorders. In Experimental Robotics [Springer Tracts in Advanced Robotics, 54], ► pp. 201 ff.
Feil-Seifer, David & Maja J. Matarić
2020. Human-Robot Interaction. In Encyclopedia of Complexity and Systems Science, ► pp. 1 ff.
Ho, Chin-Chang, Karl F. MacDorman & Z. A. D. Dwi Pramono
2008. Proceedings of the 3rd ACM/IEEE international conference on Human robot interaction, ► pp. 169 ff.
This list is based on CrossRef data as of 8 january 2025. Please note that it may not be complete. Sources presented here have been supplied by the respective publishers. Any errors therein should be reported to them.