Article published In:
Epigenetic roboticsEdited by Giorgio Metta and Luc Berthouze
[Interaction Studies 7:2] 2006
► pp. 135–169
Edited by Giorgio Metta and Luc Berthouze
[Interaction Studies 7:2] 2006
► pp. 135–169
The challenges of joint attention
This article discusses the concept of joint attention and the different skills underlying its development. Research in developmental psychology clearly states that the development of skills to understand, manipulate and coordinate attentional behavior plays a pivotal role for imitation, social cognition and the development of language. However, beside the fact that joint attention has recently received an increasing interest in the robotics community, existing models concentrate only on partial and isolated elements of these phenomena. In the line of Tomasello’s research, we argue that joint attention is much more than simultaneous looking because it implies a shared intentional relation to the world. This requires skills for attention detection, attention manipulation, social coordination and, most importantly, intentional understanding. After defining joint attention and its challenges, the current state-of-the-art of robotic and computational models relevant for this issue is discussed in relation to a developmental timeline drawn from results in child studies. From this survey, we identify open issues and challenges that still need to be addressed to understand the development of the various aspects of joint attention and conclude with the potential contribution of robotic models.
Cited by
Cited by 97 other publications
Acevedo-Valle, Juan M., Verena V. Hafner & Cecilio Angulo
2017. 2017 Joint IEEE International Conference on Development and Learning and Epigenetic Robotics (ICDL-EpiRob), ► pp. 255 ff. 
Acevedo-Valle, Juan M., Verena V. Hafner & Cecilio Angulo
2020. Social Reinforcement in Artificial Prelinguistic Development: A Study Using Intrinsically Motivated Exploration Architectures. IEEE Transactions on Cognitive and Developmental Systems 12:2 ► pp. 198 ff.
Anil Meera, Ajith, Filip Novicky, Thomas Parr, Karl Friston, Pablo Lanillos & Noor Sajid
2022. Reclaiming saliency: Rhythmic precision-modulated action and perception. Frontiers in Neurorobotics 16
Asada, M., K. Hosoda, Y. Kuniyoshi, H. Ishiguro, T. Inui, Y. Yoshikawa, M. Ogino & C. Yoshida
2009. Cognitive Developmental Robotics: A Survey. IEEE Transactions on Autonomous Mental Development 1:1 ► pp. 12 ff.
Asada, Minoru, Koh Hosoda, Hiroshi Ishiguro, Yasuo Kuniyoshi & Toshio Inui
2009. 2009 IEEE 8th International Conference on Development and Learning, ► pp. 1 ff.
Battich, Lucas, Merle Fairhurst & Ophelia Deroy
2020. Coordinating attention requires coordinated senses. Psychonomic Bulletin & Review 27:6 ► pp. 1126 ff.
Bean, Jessica L. & Inge-Marie Eigsti
2012. Assessment of joint attention in school-age children and adolescents. Research in Autism Spectrum Disorders 6:4 ► pp. 1304 ff.
Begum, Momotaz & Fakhri Karray
2011. Visual Attention for Robotic Cognition: A Survey. IEEE Transactions on Autonomous Mental Development 3:1 ► pp. 92 ff.
Begum, Momotaz, Fakhri Karray, George K. I. Mann & Raymond Gosine
2009. RO-MAN 2009 - The 18th IEEE International Symposium on Robot and Human Interactive Communication, ► pp. 200 ff.
Belhassein, Kathleen, Víctor Fernández-Castro, Amandine Mayima, Aurélie Clodic, Elisabeth Pacherie, Michèle Guidetti, Rachid Alami & Hélène Cochet
2022. Addressing joint action challenges in HRI: Insights from psychology and philosophy. Acta Psychologica 222 ► pp. 103476 ff.
Bodiroza, Sasa
2011. Using Image Depth Information for Fast Face Detection. In Towards Autonomous Robotic Systems [Lecture Notes in Computer Science, 6856], ► pp. 424 ff.
Bodiroza, Sasa, Guido Schillaci & Verena V. Hafner
2011. 2011 11th IEEE-RAS International Conference on Humanoid Robots, ► pp. 689 ff.
Boucenna, Sofiane, Antonio Narzisi, Elodie Tilmont, Filippo Muratori, Giovanni Pioggia, David Cohen & Mohamed Chetouani
2014. Interactive Technologies for Autistic Children: A Review. Cognitive Computation 6:4 ► pp. 722 ff.
Broquère, Xavier, Alberto Finzi, Jim Mainprice, Silvia Rossi, Daniel Sidobre & Mariacarla Staffa
2014. An Attentional Approach to Human–Robot Interactive Manipulation. International Journal of Social Robotics 6:4 ► pp. 533 ff.
Caccavale, Riccardo & Alberto Finzi
2017. Flexible Task Execution and Attentional Regulations in Human-Robot Interaction. IEEE Transactions on Cognitive and Developmental Systems 9:1 ► pp. 68 ff.
Cheng, Linlin, Artem V. Belopolsky & Koen V. Hindriks
2023. 2023 32nd IEEE International Conference on Robot and Human Interactive Communication (RO-MAN), ► pp. 1486 ff.
Corti, Kevin & Alex Gillespie
2016. Co-constructing intersubjectivity with artificial conversational agents: People are more likely to initiate repairs of misunderstandings with agents represented as human. Computers in Human Behavior 58 ► pp. 431 ff.
da Silva, Renato R., Claudio A. Policastro & Roseli A.F. Romero
2009. 2009 International Joint Conference on Neural Networks, ► pp. 2943 ff.
Delfing, Dalina, Karen Chin, Larissa Hentrich, Jaya Rachwani, Kathleen M. Friel, Victor Santamaria, Christine Imms & Andrew M. Gordon
2023. Assessing engagement in rehabilitation: development, validity, reliability, and responsiveness to change of the Rehabilitation Observation Measure of Engagement (ROME). Disability and Rehabilitation ► pp. 1 ff.
Ding, Keya, Chuanjiang Li, Yanwei Li & Hui Li
2023. Partner’s emotions are associated with preschoolers’ prefrontal activation under joint attention: An fNIRS evidence. Psychophysiology 60:12
Droeschel, David, Jorg Stuckler, Dirk Holz & Sven Behnke
2011. 2011 IEEE International Conference on Robotics and Automation, ► pp. 1205 ff.
Fan, Lifeng, Yixin Chen, Ping Wei, Wenguan Wang & Song-Chun Zhu
2018. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, ► pp. 6460 ff.
Ferreira, Joao Filipe & Jorge Dias
2014. Attentional Mechanisms for Socially Interactive Robots–A Survey. IEEE Transactions on Autonomous Mental Development 6:2 ► pp. 110 ff.
Fındık-Coşkunçay, Duygu & Murat Perit Çakır
2022. An investigation of the relationship between joint visual attention and product quality in collaborative business process modeling: a dual eye-tracking study. Software and Systems Modeling 21:6 ► pp. 2429 ff.
Goerick, C., H. Wersing, I. Mikhailova & M. Dunn
2005. 5th IEEE-RAS International Conference on Humanoid Robots, 2005., ► pp. 387 ff.
Gorji, Siavash & James J. Clark
2017. 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), ► pp. 3472 ff.
Haasch, A., N. Hofemann, J. Fritsch & G. Sagerer
2005. 2005 IEEE/RSJ International Conference on Intelligent Robots and Systems, ► pp. 2712 ff.
Hafner, Verena V. & Frédéric Kaplan
2008. Interpersonal Maps: How to Map Affordances for Interaction Behaviour. In Towards Affordance-Based Robot Control [Lecture Notes in Computer Science, 4760], ► pp. 1 ff.
Huang, Chien-Ming & Andrea L. Thomaz
2011. 2011 RO-MAN, ► pp. 65 ff.
Ikoma, Hibiki & Yugo Takeuchi
2023. 2023 32nd IEEE International Conference on Robot and Human Interactive Communication (RO-MAN), ► pp. 2106 ff.
Kaplan, Frederic, Pierre‐Yves Oudeyer & Benjamin Bergen
2008. Computational models in the debate over language learnability. Infant and Child Development 17:1 ► pp. 55 ff.
Karatas, Nihan, Soshi Yoshikawa, Shintaro Tamura, Sho Otaki, Ryuji Funayama & Michio Okada
2017. 2017 26th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN), ► pp. 143 ff.
Kasmarik, Kathryn, Gianluca Baldassarre, Vieri Giuliano Santucci & Jochen Triesch
2023. Guest Editorial Special Issue on Intrinsically Motivated Open-Ended Learning (IMOL). IEEE Transactions on Cognitive and Developmental Systems 15:2 ► pp. 321 ff.
Kennington, Casey
2022. 2022 31st IEEE International Conference on Robot and Human Interactive Communication (RO-MAN), ► pp. 450 ff.
Knight, Heather & Reid Simmons
2013. 2013 IEEE International Conference on Robotics and Automation, ► pp. 4350 ff.
Kuzuoka, Hideaki, Karola Pitsch, Yuya Suzuki, Ikkaku Kawaguchi, Keiichi Yamazaki, Akiko Yamazaki, Yoshinori Kuno, Paul Luff & Christian Heath
2008. Proceedings of the 2008 ACM conference on Computer supported cooperative work, ► pp. 201 ff.
Lee, Kangwoo & Hyunseung Choo
2017. Constructing Perceptual Common Ground Between Human and Robot Through Joint Attention. International Journal of Humanoid Robotics 14:03 ► pp. 1750020 ff.
León, Felipe
2021. Joint attention without recursive mindreading: On the role of second-person engagement. Philosophical Psychology 34:4 ► pp. 550 ff.
Lindsay, Grace W.
2020. Attention in Psychology, Neuroscience, and Machine Learning. Frontiers in Computational Neuroscience 14
LIU, CHAORAN, CARLOS T. ISHI, HIROSHI ISHIGURO & NORIHIRO HAGITA
2013. GENERATION OF NODDING, HEAD TILTING AND GAZING FOR HUMAN–ROBOT SPEECH INTERACTION. International Journal of Humanoid Robotics 10:01 ► pp. 1350009 ff.
Lock, Andrew & Patricia Zukow‐Goldring
2010. Preverbal Communication. In The Wiley‐Blackwell Handbook of Infant Development, ► pp. 394 ff.
Mahzoon, Hamed, Yuichiro Yoshikawa & Hiroshi Ishiguro
2016. Social Skill Acquisition Model through Face-to-Face Interaction: Local Contingency for Open-Ended Development. Frontiers in Robotics and AI 3
Mahzoon, Hamed, Yuichiro Yoshikawa & Hiroshi Ishiguro
2016. 2016 Joint IEEE International Conference on Development and Learning and Epigenetic Robotics (ICDL-EpiRob), ► pp. 308 ff.
Mahzoon, Hamed, Yuichiro Yoshikawa & Hiroshi Ishiguro
2019. Ostensive-Cue Sensitive Learning and Exclusive Evaluation of Policies: A Solution for Measuring Contingency of Experiences for Social Developmental Robot. Frontiers in Robotics and AI 6
Marin-Urias, Luis F., Emrah Akin Sisbot, Amit Kumar Pandey, Riichiro Tadakuma & Rachid Alami
2009. 2009 9th IEEE-RAS International Conference on Humanoid Robots, ► pp. 331 ff.
May, Alyxander David, Christian Dondrup & Marc Hanheide
2015. 2015 European Conference on Mobile Robots (ECMR), ► pp. 1 ff.
McDonald, Daniel G.
2008. Attention. In The International Encyclopedia of Communication,
Merrick, Kathryn
2017. Value systems for developmental cognitive robotics: A survey. Cognitive Systems Research 41 ► pp. 38 ff.
Moll, Henrike
2023. What We Do and Don’t Know About Joint Attention. Topoi
Nagai, Y.
2005. Proceedings of the 2005 IEEE International Conference on Robotics and Automation, ► pp. 2069 ff.
Neerinckx, H., P. Vos, W. Van Den Noortgate & B. Maes
2014. Temporal analysis of attentional processes in spontaneous interactions between people with profound intellectual and multiple disabilities and their support workers. Journal of Intellectual Disability Research 58:8 ► pp. 721 ff.
Noceti, Nicoletta, Alessandra Sciutti & Giulio Sandini
2015. Cognition Helps Vision: Recognizing Biological Motion Using Invariant Dynamic Cues. In Image Analysis and Processing — ICIAP 2015 [Lecture Notes in Computer Science, 9280], ► pp. 676 ff.
Oudeyer, Pierre-Yves, Frdric Kaplan & Verena V. Hafner
2007. Intrinsic Motivation Systems for Autonomous Mental Development. IEEE Transactions on Evolutionary Computation 11:2 ► pp. 265 ff.
Palinko, Oskar, Francesco Rea, Giulio Sandini & Alessandra Sciutti
2015. 2015 IEEE-RAS 15th International Conference on Humanoid Robots (Humanoids), ► pp. 318 ff.
Palinko, Oskar, Alessandra Sciutti, Yujin Wakita, Yoshio Matsumoto & Giulio Sandini
2016. 2016 IEEE-RAS 16th International Conference on Humanoid Robots (Humanoids), ► pp. 905 ff.
Panzoli, David, Adam Qureshi, Ian Dunwell, Panagiotis Petridis, Sara de Freitas & Genaro Rebolledo-Mendez
2010. Levels of Interaction (LoI): A Model for Scaffolding Learner Engagement in an Immersive Environment. In Intelligent Tutoring Systems [Lecture Notes in Computer Science, 6095], ► pp. 393 ff.
Perreira Da Silva, Matthieu, Vincent Courboulay, Armelle Prigent & Pascal Estraillier
2008. Real-Time Face Tracking for Attention Aware Adaptive Games. In Computer Vision Systems [Lecture Notes in Computer Science, 5008], ► pp. 99 ff.
Petit, Maxime, Grégoire Pointeau & Peter Ford Dominey
2016. Reasoning based on consolidated real world experience acquired by a humanoid robot. Interaction Studies. Social Behaviour and Communication in Biological and Artificial Systems 17:2 ► pp. 248 ff.
Pezzulo, Giovanni
2008. Coordinating with the Future: The Anticipatory Nature of Representation. Minds and Machines 18:2 ► pp. 179 ff.
Pfeiffer-Leßmann, Nadine & Ipke Wachsmuth
2008. Toward Alignment with a Virtual Human - Achieving Joint Attention. In KI 2008: Advances in Artificial Intelligence [Lecture Notes in Computer Science, 5243], ► pp. 292 ff.
Pfeiffer-Leßmann, Nadine & Ipke Wachsmuth
2009. Formalizing Joint Attention in Cooperative Interaction with a Virtual Human. In KI 2009: Advances in Artificial Intelligence [Lecture Notes in Computer Science, 5803], ► pp. 540 ff.
Piri, Somaye, Reza Pishghadam & Zohreh Eslami Rasekh
2021. New models of L2 achievement based on EFL learners’ joint attention and their emotional, social, and cultural capitals. International Review of Applied Linguistics in Language Teaching 59:1 ► pp. 31 ff.
Policastro, Claudio A., Giovana Zuliani, Renato R. da Silva, Vitor R. Munhoz & Roseli A.F. Romero
2008. 2008 IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence), ► pp. 1579 ff.
Pöysä-Tarhonen, Johanna, Nafisa Awwal, Päivi Häkkinen & Suzanne Otieno
2021. Joint attention behaviour in remote collaborative problem solving: exploring different attentional levels in dyadic interaction. Research and Practice in Technology Enhanced Learning 16:1
Ravindra, P., S. De Silva, Katsunori Tadano, Stephen G. Lambacher, Susantha Herath & Masatake Higashi
2009. 2009 4th International Conference on Autonomous Robots and Agents, ► pp. 601 ff.
Rolf, M., M. Hanheide & K.J. Rohlfing
2009. Attention via Synchrony: Making Use of Multimodal Cues in Social Learning. IEEE Transactions on Autonomous Mental Development 1:1 ► pp. 55 ff.
Rouanet, Pierre, Jerome Bechu & Pierre-Yves Oudeyer
2009. RO-MAN 2009 - The 18th IEEE International Symposium on Robot and Human Interactive Communication, ► pp. 1066 ff.
Rouanet, Pierre, Pierre-Yves Oudeyer & David Filliat
2009. 2009 9th IEEE-RAS International Conference on Humanoid Robots, ► pp. 391 ff.
Räsänen, Okko
2012. Computational modeling of phonetic and lexical learning in early language acquisition: Existing models and future directions. Speech Communication 54:9 ► pp. 975 ff.
Sandini, Giulio, Alessandra Sciutti & Pietro Morasso
2024. Artificial cognition vs. artificial intelligence for next-generation autonomous robotic agents. Frontiers in Computational Neuroscience 18
Sandini, Giulio, Alessandra Sciutti & Francesco Rea
2017. Movement-Based Communication for Humanoid-Human Interaction. In Humanoid Robotics: A Reference, ► pp. 1 ff.
Sandini, Giulio, Alessandra Sciutti & Francesco Rea
2019. Movement-Based Communication for Humanoid-Human Interaction. In Humanoid Robotics: A Reference, ► pp. 2169 ff.
Schauerte, B, J Richarz & G A Fink
2010. 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems, ► pp. 4638 ff.
Schillaci, Guido, Saša Bodiroža & Verena Vanessa Hafner
2013. Evaluating the Effect of Saliency Detection and Attention Manipulation in Human-Robot Interaction. International Journal of Social Robotics 5:1 ► pp. 139 ff.
Schillaci, Guido, Verena V. Hafner & Bruno Lara
2014. 4th International Conference on Development and Learning and on Epigenetic Robotics, ► pp. 130 ff.
Schnier, Christian, Karola Pitsch, Angelika Dierker & Thomas Hermann
2011. Collaboration in Augmented Reality: How to establish coordination and joint attention?. In ECSCW 2011: Proceedings of the 12th European Conference on Computer Supported Cooperative Work, 24-28 September 2011, Aarhus Denmark, ► pp. 405 ff.
Schulz, Ruth, Gordon Wyeth & Janet Wiles
2011. Lingodroids: socially grounding place names in privately grounded cognitive maps. Adaptive Behavior 19:6 ► pp. 409 ff.
Sheehan, Ben, Hyun Seung Jin & Udo Gottlieb
2020. Customer service chatbots: Anthropomorphism and adoption. Journal of Business Research 115 ► pp. 14 ff.
Siposova, Barbora & Malinda Carpenter
2019. A new look at joint attention and common knowledge. Cognition 189 ► pp. 260 ff.
Smith, Linda B., Chen Yu & Alfredo F. Pereira
2011. Not your mother’s view: the dynamics of toddler visual experience. Developmental Science 14:1 ► pp. 9 ff.
Staudte, Maria & Matthew W. Crocker
2011. Investigating joint attention mechanisms through spoken human–robot interaction. Cognition 120:2 ► pp. 268 ff.
Stjernfelt, Frederik
2012. The Evolution of Semiotic Self-Control. In The Symbolic Species Evolved [Biosemiotics, 6], ► pp. 39 ff.
Sumioka, Hidenobu, Yuichiro Yoshikawa & Minoru Asada
2010. Reproducing Interaction Contingency Toward Open-Ended Development of Social Actions: Case Study on Joint Attention. IEEE Transactions on Autonomous Mental Development 2:1 ► pp. 40 ff.
Thompson, Joe J., Nehdia Sameen, Maximilian B. Bibok & Timothy P. Racine
2013. Agnosticism gone awry: Why developmental robotics must commit to an understanding of embodiment and shared intentionality. New Ideas in Psychology 31:3 ► pp. 184 ff.
van Geert, Paul
2009. Development, Complex Dynamic Systems of. In Encyclopedia of Complexity and Systems Science, ► pp. 1872 ff.
Vazaios, Stylianos, Andreas Mallas & Michalis Xenos
2023. 2023 International Conference on Computer and Applications (ICCA), ► pp. 1 ff.
Wachsmuth, Ipke
2015. Embodied Cooperative Systems: From Tool to Partnership. In Collective Agency and Cooperation in Natural and Artificial Systems, ► pp. 63 ff.
Winfield, Alan F. T.
2018. Experiments in Artificial Theory of Mind: From Safety to Story-Telling. Frontiers in Robotics and AI 5
Winfield, Alan F. T. & Verena V. Hafner
2018. Anticipation in Robotics. In Handbook of Anticipation, ► pp. 1 ff.
Winfield, Alan F. T. & Verena V. Hafner
2019. Anticipation in Robotics. In Handbook of Anticipation, ► pp. 1587 ff.
Woolley, Anita Williams, Rosalind M. Chow, Anna T. Mayo, Christoph Riedl & Jin Wook Chang
2023. Collective Attention and Collective Intelligence: The Role of Hierarchy and Team Gender Composition. Organization Science 34:3 ► pp. 1315 ff.
Xu, Jian, Ying Ding, Yi Bu, Shuqing Deng, Chen Yu, Yimin Zou & Andrew Madden
2019. Interdisciplinary scholarly communication: an exploratory study for the field of joint attention. Scientometrics 119:3 ► pp. 1597 ff.
Yadollahi, Elmira, Wafa Johal, Ana Paiva & Pierre Dillenbourg
2018. Proceedings of the 17th ACM Conference on Interaction Design and Children, ► pp. 195 ff.
Yan, Wenjie, Elena Torta, David van der Pol, Nils Meins, Cornelius Weber, Raymond H. Cuijpers & Stefan Wermter
2013. Learning Robot Vision for Assisted Living. In Robotic Vision, ► pp. 257 ff.
Yan, Wenjie, Elena Torta, David van der Pol, Nils Meins, Cornelius Weber, Raymond H. Cuijpers & Stefan Wermter
2013. Learning Robot Vision for Assisted Living. In Image Processing, ► pp. 1232 ff.
Yucel, Z., A.A. Salah, C. Mericli & T. Mericli
2009. 2009 24th International Symposium on Computer and Information Sciences, ► pp. 242 ff.
Zhao, Zhenjie & Xiaojuan Ma
2022. Designing an Artificial Agent for Cognitive Apprenticeship Learning of Elevator Pitch in Virtual Reality. IEEE Transactions on Cognitive and Developmental Systems 14:3 ► pp. 857 ff.
This list is based on CrossRef data as of 31 march 2024. Please note that it may not be complete. Sources presented here have been supplied by the respective publishers. Any errors therein should be reported to them.