Cover not available
Article published In:
Human Robot Collaborative Intelligence: Theory and applications
Edited by Chenguang Yang, Xiaofeng Liu, Junpei Zhong and Angelo Cangelosi
[Interaction Studies 20:1] 2019
► pp. 4–24

Temporal patterns in multi-modal social interaction between elderly users and service robot

| University of the West of England
| Sheffield Hallam University
| University of Manchester
| Plymouth University

Social interaction, especially for older people living alone is a challenge currently facing human-robot interaction (HRI). There has been little research on user preference towards HRI interfaces. In this paper, we took both objective observations and participants’ opinions into account in studying older users with a robot partner. The developed dual-modal robot interface offered older users options of speech or touch screen to perform tasks. Fifteen people aged from 70 to 89 years old, participated. We analyzed the spontaneous actions of the participants, including their attentional activities and conversational activities, the temporal characteristics of these social behaviours, as well as questionnaires. It has been revealed that social engagement with the robot demonstrated by older people was no different from what might be expected towards a human partner. This study is an early attempt to reveal the social connections between human beings and a personal robot in real life.

Keywords: human-robot interaction, ageing population, multi-modal user interface, social behaviours, service robot
Article outline
  • I.Introduction
    • A.Elderly people-robots interaction
    • B.User interfaces
    • C.Research highlights
  • II.User platform for Robot-Era
  • III.Experiments and data
    • A.Participants
    • B.Experimental setup
    • C.Procedure and task
    • D.Data characteristics and annotation
  • IV.Results
    • A.Attentional behaviours – Where to look at?
    • B.Conversational behaviours – Speak or not?
    • C.Preference towards dual-modalities: “Talking” or “reading”?
    • D.PoIs
    • E.Questionnaire opinions
  • V.Conclusions
  • Acknowledgments
  • Notes
  • References
  • Sources
Published online: 15 July 2019
DOI logo
https://doi.org/10.1075/is.18042.wan
References
Al-Razgan, M., H. Al-Khalifa, M. Al-Shahrani, and H. Al-Ajmi
(2012) Touch-based mobile phone interface guidelines and design recommendations for elderly people: A survey of the literature, ser. Lecture Notes in Computer Science. Berlin Heidelberg: Springer, vol. 76661, pp. 568–574.Google Scholar
Blythe, M. A., A. F. Monk, and K. Doughty
(2005) “Socially dependable design: The challenge of ageing populations for HCI,” Interacting with Computers, vol. 17, no. 6, pp. 672–689. DOI logoGoogle Scholar
Bohus, D. and A. Rudnicky
(2009) “The ravenclaw dialog management framework: Architecture and systems,” Computer Speech and Language, vol. 23, no. 3, pp. 332–361. DOI logoGoogle Scholar
Breazealm C., C. Kidd, A. Thomaz, G. Hoffman, and M. Berlin
(2005) “Effects of nonverbal communication on efficiency and robustness in human-robot teamwork,” in Proc. IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 708–713.Google Scholar
Cavallo, F., R. Esposito, R. Limosani, A. Manzi, R. Bevilacqua, E. Felici, A. D. Nuovo, A. Cangelosi, F. Lattanzio, and P. Dario
(2018) “Robotic services acceptance in smart environments with older adults: User satisfaction and acceptability study,” J Med Internet Res, vol. 20, no. 9. DOI logoGoogle Scholar
Di Nuovo, A., F. Broz, T. Belpaeme, A. Cangelosi, F. Cavallo, R. Esposito, and P. Dario
(2014) “A web based multi-modal interface for elderly users of the Robot-Era multi-robot services,” in Proc. IEEE International Conference on Systems, Man and Cybernetics, pp. 2186–2191.Google Scholar
Hare, B. and M. Tomasello
(2005) “Human-like social skills in dogs?” Trends in Cognitive Sciences, vol. 9, no. 9, pp. 439–444. DOI logoGoogle Scholar
Kipp, M.
(2014) Handbook of Corpus Phonology. Oxford University Press, ch. ANVIL: A Universal Video Research Tool, pp. 420–436.Google Scholar
Knapp, M. L., J. A. Hall, and T. G. Horgan
(2013) Nonverbal communication in human interaction, 8th ed. Boston, USA: Wadsworth 2013.Google Scholar
Lemon, O.
(2004) “Context-sensitive speech recognition in Information-State Update dialogue systems: results for the grammar switching approach,” in Proc. Eighth Workshop on the Semantics and Pragmatics of Dialogue, pp. 49–55.Google Scholar
Linert, J. and P. Kopacek
(2016) “Robots for education (edutainment),” in IFAC-PapersOnLine, Elsevier, Ed., vol. 49, no. 29, pp. 24–29.Google Scholar
Mayer, P., C. Beck, and P. Panek
(2012) “Examples of multimodal user interfaces for socially assistive robots in ambient assisted environments,” in Proc. IEEE International Conference on Cognitive Infocommunications, pp. 401–406.Google Scholar
Morency, L.-P. and T. Darrell
(2004) “From conversational tooltips to grounded discourse: Head post tracking in interactive dialog systems,” in ACM International Conference on Multimodal Interaction, pp. 32–37.Google Scholar
Mutlu, B., T. Shiwa, T. Kanda, H. Ishiguro, and N. Hagita
(2009) “Footing in human-robot conversations: How robots might shape participant roles using gaze cues,” in Proc. ACM/IEEE International Conference on Human Robot Interaction, pp. 61–68.Google Scholar
Portet, F., M. Vacher, C. Golanski, C. Roux, and B. Meillon
(2013) “Design and evaluation of a smart home voice interface for the elderly: acceptability and objection aspects,” Personal and Ubiquitous Computing, vol. 17, no. 1, pp. 127–144. DOI logoGoogle Scholar
“
Robot-Era project: Implementation and integration of advanced robotic systems and intelligent environments in real scenarios for the ageing population,” FP7-ICT-Challenge 5: ICT for Health, Ageing Well, Inclusion and Governance. Grant agreement number 288899.
Schneider, J., S. Irgenfried, W. Stork, and H. Wörn
(2015) “A multimodal human machine interface for a robotic mobility aid,” in Proc. IEEE International Conference on Automation, Robotics and Applications, pp. 289–294.Google Scholar
Sehili, M. A., F. Yang, and L. Devillers
(2014) “Attention detection in elderly people-robot spoken interaction,” in Proc. ICMI Workshop on Multimodal, Multi-Party, Real-World Human-Robot Interaction, pp. 7–12.Google Scholar
Sheikhi, S. and J.-M. Odobez
(2014) “Combining dynamic head pose-gaze mapping with the robot conversational state for attention recognition in human-robot interactions,” Pattern Recognition Letters.Google Scholar
Smarr, C., A. Prakash, J. Beer, T. Mitzner, C. Kemp, and W. Rogers
(2012) “Older adults preferences for and acceptance of robot assistance for everyday living tasks,” in Proc. Human Factors and Ergonomics Society Annual Meeting, pp. 153–157. DOI logoGoogle Scholar
Staudte, M. and M. Crocker
(2009) “Visual attention in spoken human-robot interaction,” in Proc. ACM/IEEE International Conference on Human-Robot Interaction, pp. 77–84.Google Scholar
Tang, D., B. Yusuf, J. Botzheim, N. Kubota, and C. Chan
(2015) “A novel multimodal communication framework using robot partner for aging population,” Expert Systems with Applications, vol. 42, no. 9, pp. 4540–4555. DOI logoGoogle Scholar
United Nations
(2017, October). “Population facts.”Google Scholar
Wang, N., F. Broz, A. Di Nuovo, T. Belpaeme, and A. Cangelosi
(2016) Recent Advances in Nonlinear Speech Processing, ser. 2190–3018. Springer International Publishing, vol. 481, ch. A user-centric design of service robots speech interface for the elderly, pp. 275–283. DOI logoGoogle Scholar
Yamazaki, A., K. Yamazaki, Y. Kuno, M. Burdelski, M. Kawashima, and H. Kuzuoka
(2008) “Precision timing in human-robot interaction: Coordination of head movement and utterance,” in Proc. SIGCHI Conference on Human Factors in Computing Systems, pp. 131–140.Google Scholar
Yu, C., M. Scheutz, and P. Schermerhorn
(2010) “Investigating multimodal real-time patterns of joint attention in an HRI word learning task,” in Proc. ACM/IEEE International Conference on Human-Robot Interaction, pp. 309–316.Google Scholar

Sources

(
2015) [Online]. Available: [URL]
Cited by

Cited by 5 other publications

Carradore, Marco
2021. People’s Attitudes Towards the Use of Robots in the Social Services: A Multilevel Analysis Using Eurobarometer Data. International Journal of Social Robotics DOI logo
Conti, Daniela, Santo Di Nuovo & Alessandro Di Nuovo
2021. A Brief Review of Robotics Technologies to Support Social Interventions for Older Users. In Human Centred Intelligent Systems [Smart Innovation, Systems and Technologies, 189], ► pp. 221 ff. DOI logo
Wang, J., Y. Fu, V. Lou, S.Y. Tan & E. Chui
2021. A systematic review of factors influencing attitudes towards and intention to use the long-distance caregiving technologies for older adults. International Journal of Medical Informatics 153 ► pp. 104536 ff. DOI logo
Wang, Ning, Chuize Chen & Alessandro Di Nuovo
2021. A Framework of Hybrid Force/Motion Skills Learning for Robots. IEEE Transactions on Cognitive and Developmental Systems 13:1 ► pp. 162 ff. DOI logo
Zhuang, Liang, Ching-Hsien Hsu & Priyan Malarvizhi Kumar
2021. IoT Based Multimodal Social Interaction Activity Framework for the Physical Education System. Wireless Personal Communications DOI logo

This list is based on CrossRef data as of 15 april 2022. Please note that it may not be complete. Sources presented here have been supplied by the respective publishers. Any errors therein should be reported to them.