Article published In:
Psychological Benchmarks of Human–Robot InteractionEdited by Peter H. Kahn, Jr. and Karl F. MacDorman
[Interaction Studies 8:3] 2007
► pp. 423–439
Edited by Peter H. Kahn, Jr. and Karl F. MacDorman
[Interaction Studies 8:3] 2007
► pp. 423–439
Benchmarks for evaluating socially assistive robotics
Socially assistive robotics (SAR) is a growing area of research. Evaluating SAR systems presents novel challenges. Using a robot for a socially assistive task can have various benefits and ethical implications. Many questions are important to understanding whether a robot is effective for a given application domain. This paper describes several benchmarks for evaluating SAR systems. There exist numerous methods for evaluating the many factors involved in a robot’s design. Benchmarks from psychology, anthropology, medicine, and human–robot interaction are proposed as measures of success in evaluating a given SAR system and its impact on the user and broader population.
Cited by
Cited by 84 other publications
Adlakha, Deepansha & Dolly Sharma
2022. Human‐Robot Interaction: An Artificial Cognition‐Based Study for Criminal Investigations. In Digital Forensics and Internet of Things, ► pp. 163 ff. 
Aguiar Noury, Gabriel, Hannah Bradwell, Serge Thill & Ray Jones
2019. User-defined challenges and desiderata for robotics and autonomous systems in health and social care settings. Advanced Robotics 33:7-8 ► pp. 309 ff.
Ahmad, Sayed Fayaz, Mohd. Khairil Rahmat, Muhammad Shujaat Mubarik, Muhammad Mansoor Alam & Syed Irfan Hyder
2021. Artificial Intelligence and Its Role in Education. Sustainability 13:22 ► pp. 12902 ff.
Ajoudani, Arash, Andrea Maria Zanchettin, Serena Ivaldi, Alin Albu-Schäffer, Kazuhiro Kosuge & Oussama Khatib
2018. Progress and prospects of the human–robot collaboration. Autonomous Robots 42:5 ► pp. 957 ff.
Aly, Amir, Sascha Griffiths & Francesca Stramandinoli
2017. Metrics and benchmarks in human-robot interaction: Recent advances in cognitive robotics. Cognitive Systems Research 43 ► pp. 313 ff.
Ammon, Sabine & Martin Meister
2019. Roboter, humanoide. In Mensch-Maschine-Interaktion, ► pp. 297 ff.
Angulo, Cecilio, Sammy Pfeiffer, Ricardo Tellez, Guillem Alenyà, Stefano Chessa, Juan Antonio Alvarez García & Paolo Barsocchi
2015. Evaluating the use of robots to enlarge AAL services1. Journal of Ambient Intelligence and Smart Environments 7:3 ► pp. 301 ff.
Baraka, Kim, Patrícia Alves-Oliveira & Tiago Ribeiro
2020. An Extended Framework for Characterizing Social Robots. In Human-Robot Interaction [Springer Series on Bio- and Neurosystems, 12], ► pp. 21 ff.
Beer, Jenay M., Arthur D. Fisk & Wendy A. Rogers
2015. Commanding Home Robots. Proceedings of the Human Factors and Ergonomics Society Annual Meeting 59:1 ► pp. 70 ff.
Bodine, Cathy, Levin Sliker, Michael Marquez, Cecilia Clark, Brian Burne & Jim Sandstrum
2017. 8. Social Assistive Robots for Children with Complex Disabilities. In Robotic Assistive Technologies [Rehabilitation Science in Practice Series, ], ► pp. 261 ff.
Bonarini, Andrea
2020. Communication in Human-Robot Interaction. Current Robotics Reports 1:4 ► pp. 279 ff.
Brenčič, Neja Samar, Marius Dragoi, Irina Mocanu & Tomasz Winiarski
2020. Intuitive and Intelligent Solutions for Elderly Care. In Digital Health in Focus of Predictive, Preventive and Personalised Medicine [Advances in Predictive, Preventive and Personalised Medicine, 12], ► pp. 101 ff.
Broadbent, E., R. Stafford & B. MacDonald
2009. Acceptance of Healthcare Robots for the Older Population: Review and Future Directions. International Journal of Social Robotics 1:4 ► pp. 319 ff.
Broadbent, Elizabeth, Rie Tamagawa, Ngaire Kerse, Brett Knock, Anna Patience & Bruce MacDonald
2009. RO-MAN 2009 - The 18th IEEE International Symposium on Robot and Human Interactive Communication, ► pp. 645 ff.
Caine, Kelly, Selma Šabanovic & Mary Carter
2012. Proceedings of the seventh annual ACM/IEEE international conference on Human-Robot Interaction, ► pp. 343 ff.
Cerulo, Karen A.
2009. Nonhumans in Social Interaction. Annual Review of Sociology 35:1 ► pp. 531 ff.
Cerulo, Karen A.
2011. Social Interaction: Do Non-humans Count?. Sociology Compass 5:9 ► pp. 775 ff.
Chang, Rebecca Cherng-Shiow, Hsi-Peng Lu & Peishan Yang
2018. Stereotypes or golden rules? Exploring likable voice traits of social robots as active aging companions for tech-savvy baby boomers in Taiwan. Computers in Human Behavior 84 ► pp. 194 ff.
Coeckelbergh, Mark, Cristina Pop, Ramona Simut, Andreea Peca, Sebastian Pintea, Daniel David & Bram Vanderborght
2016. A Survey of Expectations About the Role of Robots in Robot-Assisted Therapy for Children with ASD: Ethical Acceptability, Trust, Sociability, Appearance, and Attachment. Science and Engineering Ethics 22:1 ► pp. 47 ff.
Compagna, Diego & Ivo Boblan
2015. Case-Sensitive Methods for Evaluating HRI from a Sociological Point of View. In Social Robotics [Lecture Notes in Computer Science, 9388], ► pp. 155 ff.
Compagna, Diego, Manuela Marquardt & Ivo Boblan
2016. Introducing a Methodological Approach to Evaluate HRI from a Genuine Sociological Point of View. In Cultural Robotics [Lecture Notes in Computer Science, 9549], ► pp. 55 ff.
Corti, Kevin & Alex Gillespie
2016. Co-constructing intersubjectivity with artificial conversational agents: People are more likely to initiate repairs of misunderstandings with agents represented as human. Computers in Human Behavior 58 ► pp. 431 ff.
Devaraj, Uthirakumar & Balamurugan Elumalai
2022. Robot Therapy. In Handbook of Research on Lifestyle Sustainability and Management Solutions Using AI, Big Data Analytics, and Visualization [Advances in Computational Intelligence and Robotics, ], ► pp. 137 ff.
Elbeleidy, Saad, Terran Mott & Tom Williams
2022. 2022 17th ACM/IEEE International Conference on Human-Robot Interaction (HRI), ► pp. 577 ff.
Esposito, Anna, Antonietta M. Esposito & Carl Vogel
2015. Needs and challenges in human computer interaction for processing social emotional information. Pattern Recognition Letters 66 ► pp. 41 ff.
Esposito, Anna & Lakhmi C. Jain
2016. Modeling Social Signals and Contexts in Robotic Socially Believable Behaving Systems. In Toward Robotic Socially Believable Behaving Systems - Volume II [Intelligent Systems Reference Library, 106], ► pp. 5 ff.
Fazlollahi, Farimah & Katherine J. Kuchenbecker
2023. Haptify: A Measurement-Based Benchmarking System for Grounded Force-Feedback Devices. IEEE Transactions on Robotics 39:2 ► pp. 1622 ff.
Feil-Seifer, David & Maja Mataric
2011. Socially Assistive Robotics. IEEE Robotics & Automation Magazine 18:1 ► pp. 24 ff.
Feil-Seifer, David & Maja J. Mataric
2008. RO-MAN 2008 - The 17th IEEE International Symposium on Robot and Human Interactive Communication, ► pp. 328 ff.
Feil-Seifer, David & Maja J. Matarić
2009. Human Robot Interaction. In Encyclopedia of Complexity and Systems Science, ► pp. 4643 ff.
Feil-Seifer, David & Maja J. Matarić
2009. Toward Socially Assistive Robotics for Augmenting Interventions for Children with Autism Spectrum Disorders. In Experimental Robotics [Springer Tracts in Advanced Robotics, 54], ► pp. 201 ff.
Feil-Seifer, David & Maja J. Matarić
2020. Human-Robot Interaction. In Encyclopedia of Complexity and Systems Science, ► pp. 1 ff.
Feth, Daniela
2011. Haptic Human–Robot Collaboration: Comparison of Robot Partner Implementations in Terms of Human-Likeness and Task Performance. Presence: Teleoperators and Virtual Environments 20:2 ► pp. 173 ff.
Fiederer, Lukas D. J., Martin Völker, Robin T. Schirrmeister, Wolfram Burgard, Joschka Boedecker & Tonio Ball
2019. Hybrid Brain-Computer-Interfacing for Human-Compliant Robots: Inferring Continuous Subjective Ratings With Deep Regression. Frontiers in Neurorobotics 13
Fortuna, Karen L., Maria Venegas, Emre Umucu, George Mois, Robert Walker & Jessica M. Brooks
2019. The Future of Peer Support in Digital Psychiatry: Promise, Progress, and Opportunities. Current Treatment Options in Psychiatry 6:3 ► pp. 221 ff.
Franke, Annette, Elmar Nass, Anna-Kathleen Piereth, Annabel Zettl & Christian Heidl
2021. Implementation of Assistive Technologies and Robotics in Long-Term Care Facilities: A Three-Stage Assessment Based on Acceptance, Ethics, and Emotions. Frontiers in Psychology 12
Fridin, Marina
2014. Kindergarten social assistive robot: First meeting and ethical issues. Computers in Human Behavior 30 ► pp. 262 ff.
Gadde, Prathik, Hadi Kharrazi, Himalaya Patel & Karl F. MacDorman
2011. Toward Monitoring and Increasing Exercise Adherence in Older Adults by Robotic Intervention: A Proof of Concept Study. Journal of Robotics 2011 ► pp. 1 ff.
Groten, Raphaela, Daniela Feth, Angelika Peer & Martin Buss
2012. Psychological Experiments in Haptic Collaboration Research. In Immersive Multimodal Interactive Presence [Springer Series on Touch and Haptic Systems, ], ► pp. 65 ff.
Han, Isla Xi, Forrest Meggers & Stefana Parascho
2021. Bridging the collectives: A review of collective human–robot construction. International Journal of Architectural Computing 19:4 ► pp. 512 ff.
Hersh, Marion
2015. Overcoming Barriers and Increasing Independence – Service Robots for Elderly and Disabled People. International Journal of Advanced Robotic Systems 12:8 ► pp. 114 ff.
Ho, Chin-Chang, Karl F. MacDorman & Z. A. D. Dwi Pramono
2008. Proceedings of the 3rd ACM/IEEE international conference on Human robot interaction, ► pp. 169 ff.
Huber, Andreas, Astrid Weiss & Marjo Rauhala
2016. 2016 11th ACM/IEEE International Conference on Human-Robot Interaction (HRI), ► pp. 367 ff.
Insaurralde, Carlos C. & Thilo Kaupisch
2016. 2016 IEEE Aerospace Conference, ► pp. 1 ff.
Insaurralde, Carlos C. & Thilo Kaupisch
2018. 2018 IEEE Aerospace Conference, ► pp. 1 ff.
Jahan, Farha, Weiqing Sun, Quamar Niyaz & Mansoor Alam
2020. Security Modeling of Autonomous Systems. ACM Computing Surveys 52:5 ► pp. 1 ff.
JEREMY, (ZHENG) LI
2017. NEW TECHNOLOGY IN COMPUTER AIDED DESIGN, COMPUTER AIDED MANUFACTURING, COMPUTER AIDED ENGINEERING ANALYTIC METHODOLOGY, AUTOMATED AND HIGH SPEED MECHANICAL SYSTEM. i-manager's Journal on Computer Science 5:3 ► pp. 6 ff.
Kipp, Andreas & Sebastian Schneider
2017. Applied Social Robotics—Building Interactive Robots with LEGO Mindstorms. In Robotics in Education [Advances in Intelligent Systems and Computing, 457], ► pp. 29 ff.
Le, Hoang H., Martin J. Loomes & Rui C.V. Loureiro
2016. 2016 12th International Conference on Intelligent Environments (IE), ► pp. 62 ff.
Li, Zheng (Jeremy)
2015. Robotic System Design and Development for Medical Treatment, Surgery, and Automated Production Applications. In Handbook of Research on Advancements in Robotics and Mechatronics [Advances in Computational Intelligence and Robotics, ], ► pp. 851 ff.
Lima, Maria R., Maitreyee Wairagkar, Manish Gupta, Ferdinando Rodriguez y Baena, Payam Barnaghi, David J. Sharp & Ravi Vaidyanathan
2022. Conversational Affective Social Robots for Ageing and Dementia Support. IEEE Transactions on Cognitive and Developmental Systems 14:4 ► pp. 1378 ff.
MacDorman, Karl F., Sandosh K. Vasudevan & Chin-Chang Ho
2009. Does Japan really have robot mania? Comparing attitudes by implicit and explicit measures. AI & SOCIETY 23:4 ► pp. 485 ff.
Matsuzaki, Hironori & Gesa Lindemann
2016. The autonomy-safety-paradox of service robotics in Europe and Japan: a comparative analysis. AI & SOCIETY 31:4 ► pp. 501 ff.
McColl, D., Wing-Yue Geoffrey Louie & G. Nejat
2013. Brian 2.1: A socially assistive robot for the elderly and cognitively impaired. IEEE Robotics & Automation Magazine 20:1 ► pp. 74 ff.
Mitchell, Wade J., Chin-Chang Ho, Himalaya Patel & Karl F. MacDorman
2011. Does social desirability bias favor humans? Explicit–implicit evaluations of synthesized speech support a new HCI model of impression management. Computers in Human Behavior 27:1 ► pp. 402 ff.
Mois, George & Jenay M. Beer
2020. Robotics to support aging in place. In Living with Robots, ► pp. 49 ff.
Mois, George & Jenay M. Beer
2020. The Role of Healthcare Robotics in Providing Support to Older Adults: a Socio-ecological Perspective. Current Geriatrics Reports 9:2 ► pp. 82 ff.
Nikolopoulos, Chris, Deitra Kuester, Mark Sheehan, Sneha Dhanya, William Herring, Andrew Becker & Lauralyn Bogart
2010. Socially Assistive Robots and Autism. Solid State Phenomena 166-167 ► pp. 315 ff.
Nikolopoulos, Chris, Deitra Kuester, Mark Sheehan & Sneha Dhanya
2011. Investigation on Requirements of Robotic Platforms to Teach Social Skills to Individuals with Autism. In Human-Robot Personal Relationships [Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering, 59], ► pp. 65 ff.
Onnasch, Linda & Eileen Roesler
2021. A Taxonomy to Structure and Analyze Human–Robot Interaction. International Journal of Social Robotics 13:4 ► pp. 833 ff.
Park, Chung Hyuk, Hifza Javed & Myounghoon Jeon
2019. Consensus-Based Human-Agent Interaction Model for Emotion Regulation in ASD. In HCI International 2019 - Posters [Communications in Computer and Information Science, 1032], ► pp. 295 ff.
Rabbitt, Sarah M., Alan E. Kazdin & Brian Scassellati
2015. Integrating socially assistive robotics into mental healthcare interventions: Applications and recommendations for expanded use. Clinical Psychology Review 35 ► pp. 35 ff.
Rahaman, Ashiqur & Byungki Kim
2020. 2020 20th International Conference on Control, Automation and Systems (ICCAS), ► pp. 1120 ff.
Rossi, Silvia, Alessandra Rossi & Kerstin Dautenhahn
2020. The Secret Life of Robots: Perspectives and Challenges for Robot’s Behaviours During Non-interactive Tasks. International Journal of Social Robotics 12:6 ► pp. 1265 ff.
Salehuddin, Muhammad, Suprijanto & Augie Widyotriatmo
2017. 2017 International Conference on Smart Cities, Automation & Intelligent Computing Systems (ICON-SONICS), ► pp. 147 ff.
Schulz, Trenton & Jo Herstad
2017. [Electronic Workshops in Computing, ],
Schulz, Trenton, Jo Herstad & Harald Holone
2018. Privacy at Home: An Inquiry into Sensors and Robots for the Stay at Home Elderly. In Human Aspects of IT for the Aged Population. Applications in Health, Assistance, and Entertainment [Lecture Notes in Computer Science, 10927], ► pp. 377 ff.
Schulze, Hartmut, Simone Kauffeld & Alexandra Tanner
2022. Soziale Roboter auf dem Weg in die Praxis. Gruppe. Interaktion. Organisation. Zeitschrift für Angewandte Organisationspsychologie (GIO) 53:3 ► pp. 267 ff.
Sengupta, Rounak & Vivek Kant
2022. Holistic Asset Performance Management Systems: Speculative design for Industrial APM systems and human work experience in an Industry 5.0 setting. INCOSE International Symposium 32:S1 ► pp. 182 ff.
Shukla, Jainendra, Miguel Barreda-Angeles, Joan Oliver & Domenec Puig
2017. 2017 26th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN), ► pp. 62 ff.
Subin, E. K., Ashik Hameed & A. P. Sudheer
2017. Proceedings of the Advances in Robotics, ► pp. 1 ff.
Tan, Yeow Kee, Alvin Wong, Anthony Wong, Tran Anh Dung, Adrian Tay, Dilip Limbu Kumar, Tran Huy Dat, Weng Zheng Ng, Rui Yan & Benedict Tay
2013. Evaluation of the Pet Robot CuDDler Using Godspeed Questionnaire. In Inclusive Society: Health and Wellbeing in the Community, and Care at Home [Lecture Notes in Computer Science, 7910], ► pp. 102 ff.
Thoduka, Santosh & Nico Hochgeschwender
2021. Benchmarking Robots by Inducing Failures in Competition Scenarios. In Digital Human Modeling and Applications in Health, Safety, Ergonomics and Risk Management. AI, Product and Service [Lecture Notes in Computer Science, 12778], ► pp. 263 ff.
Trautman, Pete, Jeremy Ma, Richard M. Murray & Andreas Krause
2015. Robot navigation in dense human crowds: Statistical models and experimental studies of human–robot cooperation. The International Journal of Robotics Research 34:3 ► pp. 335 ff.
van der Zant, Tijn & Luca Iocchi
2011. RoboCup
Home: Adaptive Benchmarking of Robot Bodies and Minds. In Social Robotics [Lecture Notes in Computer Science, 7072], ► pp. 214 ff.
Home: Adaptive Benchmarking of Robot Bodies and Minds. In Social Robotics [Lecture Notes in Computer Science, 7072], ► pp. 214 ff. Vasylkiv, Yurii, Ricardo Ragel, Javier Ponce-Chulani, Luis Merino, Eleanor Sandry, Heike Brock, Keisuke Nakamura, Irani Pourang & Randy Gomez
2021. Design and Development of a Teleoperation System for Affective Tabletop Robot Haru. In Social Robotics [Lecture Notes in Computer Science, 13086], ► pp. 564 ff.
Vazquez-Santacruz, Eduardo & Mariano Gamboa-Zuniga
2013. 2013 10th International Conference on Electrical Engineering, Computing Science and Automatic Control (CCE), ► pp. 163 ff.
Welfare, Katherine S., Matthew R. Hallowell, Julie A. Shah & Laurel D. Riek
2019. 2019 14th ACM/IEEE International Conference on Human-Robot Interaction (HRI), ► pp. 75 ff.
Werner, Franz
2020. A Survey on Current Practices in User Evaluation of Companion Robots. In Human-Robot Interaction [Springer Series on Bio- and Neurosystems, 12], ► pp. 65 ff.
Yamaji, Yuto, Taisuke Miyake, Yuta Yoshiike, P. Ravindra S. De Silva & Michio Okada
2011. STB: Child-Dependent Sociable Trash Box. International Journal of Social Robotics 3:4 ► pp. 359 ff.
Zheng Li, Jeremy
2015. Magnetic Sealing System. In CAD, 3D Modeling, Engineering Analysis, and Prototype Experimentation, ► pp. 129 ff.
Čaić, Martina, João Avelino, Dominik Mahr, Gaby Odekerken-Schröder & Alexandre Bernardino
2020. Robotic Versus Human Coaches for Active Aging: An Automated Social Presence Perspective. International Journal of Social Robotics 12:4 ► pp. 867 ff.
Čaić, Martina, Dominik Mahr & Gaby Oderkerken-Schröder
2019. Value of social robots in services: social cognition perspective. Journal of Services Marketing 33:4 ► pp. 463 ff.
Čaić, Martina, Gaby Odekerken-Schröder & Dominik Mahr
2018. Service robots: value co-creation and co-destruction in elderly care networks. Journal of Service Management 29:2 ► pp. 178 ff.
This list is based on CrossRef data as of 23 may 2023. Please note that it may not be complete. Sources presented here have been supplied by the respective publishers. Any errors therein should be reported to them.