Reasoning based on consolidated real world experience acquired by a humanoid robot
The development of reasoning systems exploiting expert knowledge from interactions with humans is a non-trivial problem, particularly when considering how the information can be coded in the knowledge representation. For example, in human development, the acquisition of knowledge at one level requires the consolidation of knowledge from lower levels. How is the accumulated experience structured to allow the individual to apply knowledge to new situations, allowing reasoning and adaptation? We investigate how this can be done automatically by an iCub that interacts with humans to acquire knowledge via demonstration. Once consolidated, this knowledge is used in further acquisitions of experience concerning preconditions and consequences of actions. Finally, this knowledge is translated into rules that allow reasoning and planning for novel problem solving, including a Tower of Hanoi scenario. We thus demonstrate proof of concept for an interaction system that uses knowledge acquired from human interactions to reason about new situations.
Article outline
- 1.Introduction
- 2.Robot system description
- 2.1iCub
- 2.2ReacTable
- 2.3Object Properties Collector
- 2.4Interaction Supervisor
- 3.Autobiographical memory and reasoning
- 3.1Episodic-Like Memory
- 3.2Semantic memory
- 3.3Retro reasoning
- 3.4Level 3 reasoning
- 3.5From interaction to successive level of representation
- 4.Planning and goal directed reasoning
- 4.1Planning Domain Definition Language (PDDL) framework
- 5.Experiments
- 5.1Knowledge components to run the experiments
- 5.2Experiment 1: Learning rules about spatial movement – proof of concept
- 5.3Experiment 2: The Table of Hanoi
- 6.Discussion and conclusion
- Acknowledgements
-
References
References
Broz, F., Nehaniv, C. L., Kose-Bagci, H., & Dautenhahn, K.
(
2012)
Interaction Histories and Short Term Memory: Enactive Development of Turn-taking Behaviors in a Childlike Humanoid Robot.
Computing Research Repository. Artificial Intelligence; Adaptation and Self-Organizing Systems.
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Cangelosi, A., Metta, G., Sagerer, G., Nolfi, S., Nehaniv, C., Fischer, K., … Zeschel, A.
(
2010)
Integration of Action and Language Knowledge: A Roadmap for Developmental Robotics.
IEEE Transactions on Autonomous Mental Development, 2(1), 167–195.
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Carey, S.
(
2009)
The Origin of Concepts. Oxford University Press.
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Carey, S., & Xu, F.
(
2001)
Infants’ knowledge of objects: beyond object files and object tracking.
Cognition, 801(1–21), 179–213.
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Cheng, P. W.
(
1996)
From Covariation to Causation: A Causal Power Theory.
Psychological Review, 104(1), 367–405.
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Crangle, C., & Suppes, P.
(
1994)
Language and Learning for Robots. Center for the Study of Language and Information.
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Demiris, Y., & Khadhouri, B.
(
2006)
Content-based control of goal-directed attention during human action perception. In
ROMAN 2006 – The 15th IEEE International Symposium on Robot and Human Interactive Communication (pp. 226–231). IEEE.
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Dindo, H., & Schillaci, G.
(
2010)
An adaptive probabilistic approach to goal-level imitation learning. IEEE/RSJ 2010 International Conference on Intelligent Robots and Systems, IROS 2010 – Conference Proceedings, 4452–4457.
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Dominey, P. F.
(
2003)
Learning Grammatical Constructions in a Miniature Language from Narrated Video Events. In
25th Annual Meeting of the Cognition Science Society, Boston.
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Dominey, P. F., & Dodane, C.
(
2004)
Indeterminacy in language acquisition: the role of child directed speech and joint attention.
Journal of Neurolinguistics, 17(2–3), 121–145.
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Dominey, P. F., Mallet, A., & Yoshida, E.
(
2007a)
Progress in Programming the HRP-2 Humanoid Using Spoken Language. In Proceedings 2007 IEEE International Conference on Robotics and Automation (pp. 2169–2174). IEEE.
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Dominey, P. F., Mallet, A., & Yoshida, E.
(
2007b)
Real-time cooperative behavior acquisition by a humanoid apprentice. In 2007 7th IEEE-RAS International Conference on Humanoid Robots (pp. 270–275). IEEE.
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Dominey, P. F., Mallet, A., & Yoshida, E.
(
2009)
Real-time spoken-language programming for cooperative interaction with a humanoid apprentice.
International Journal of Humanoid Robotics, 6(1), 147–171.
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Dore, A., Cattoni, A. F., & Regazzoni, C. S.
(
2010)
Interaction Modeling and Prediction in Smart Spaces: A Bio-Inspired Approach Based on Autobiographical Memory.
IEEE Transactions on Systems, Man, and Cybernetics – Part A: Systems and Humans, 40(1), 1191–1205.
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Doshi, F., & Roy, N.
(
2008)
Spoken language interaction with model uncertainty: an adaptive human – robot interaction system.
Connection Science, 20(1), 299–318.
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Fern, A., Givan, R., & Siskind, J. M.
(
2002)
Specific-to-General Learning for Temporal Events with Application to Learning Event Definitions from Video.
J. Artificial Intelligence Res., 103–129.
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
G. L. Drescher.
(
1991)
Made-up minds: a constructivist approach to artificial intelligence. MIT press.
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Gergely, G., & Csibra, G.
(
2003)
Teleological reasoning in infancy: the naı̈ve theory of rational action.
Trends in Cognitive Sciences, 7(1), 287–292.
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Gergely, G., & Csibra, G.
(
2005)
The social construction of the cultural mind: Imitative learning as a mechanism of human pedagogy.
Interaction Studies, 6(1), 463–481.
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Gergely, G., & Csibra, G.
(
2006)
Sylvia’s recipe: The role of imitation and pedagogy in the transmission of cultural knowledge.
Roots of Human Sociality: Culture, Cognition, and Human Interaction, 229–255.
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Gergely, G., Nádasdy, Z., Csibra, G., & Bíró, S.
(
1995)
Taking the intentional stance at 12 months of age.
Cognition, 56(1), 165–193.
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Gori, I., Pattacini, U., Nori, F., Metta, G., & Sandini, G.
(
2012)
DForC: A real-time method for reaching, tracking and obstacle avoidance in humanoid robots. In 2012 12th IEEE-RAS International Conference on Humanoid Robots (Humanoids 2012) (pp. 544–551). IEEE.
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Gorniak, P., & Roy, D.
(
2004)
Grounded Semantic Composition for Visual Scenes.
Journal of Artificial Intelligence Research, 211, 429–470.
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Guez, A., Silver, D., & Dayan, P.
(
2013)
Scalable and efficient bayes-adaptive reinforcement learning based on Monte-Carlo tree search.
Journal of Artificial Intelligence Research, 481, 841–883.
Hayes-Roth, F.
(
1997)
June 15.
Artificial Intelligence: What Works and What Doesn’t? AI Magazine.
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Helmert, M.
(
2009)
Concise finite-domain representations for PDDL planning tasks.
Artificial Intelligence, 1731(5–6), 503–535.
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Ho, W. C., Dautenhahn, K., Lim, M. Y., Vargas, P. A., Aylett, R., & Enz, S.
(
2009)
An initial memory model for virtual and robot companions supporting migration and long-term interaction. In
RO-MAN 2009 – The 18th IEEE International Symposium on Robot and Human Interactive Communication (pp. 277–284). IEEE.
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Kalkan, S., Dag, N., Yürüten, O., Borghi, A. M., & Sahin, E.
Kaplan, F., & Hafner, V. V.
(
2006)
The challenges of joint attention.
Interaction Studies, 7(1), 135–169.
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Kulick, J., Toussaint, M., Lang, T., Lopes, M., Sud-ouest, I. B., & Bat, A.
(
2013)
Active Learning for Teaching a Robot Grounded Relational Symbols at Stuttgart. In Proceedings of the Twenty-Third international joint conference on Artificial Intelligence.
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Lallee, S., Lemaignan, S., Lenz, A., Melhuish, C., Natale, L., Skachek, S., … Dominey, P. F.
(
2010)
Towards a platform-independent cooperative human-robot interaction system: I. Perception. In 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems (pp. 4444–4451). IEEE.
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Lallee, S., Madden, C., Hoen, M., & Dominey, P. F.
(
2010)
Linking language with embodied and teleological representations of action for humanoid cognition.
Frontiers in Neurorobotics, 4, 8.
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Lallee, S., Pattacini, U., Boucher, J. D., Lemaignan, S., Lenz, A., Melhuish, C., … Dominey, P. F.
(
2011)
Towards a platform-independent cooperative human-robot interaction system: II. Perception, execution and imitation of goal directed actions. In 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems (pp. 2895–2902). IEEE.
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Lallee, S., Pattacini, U., Lemaignan, S., Lenz, A., Melhuish, C., Natale, L., … Dominey, P. F.
(
2012)
Towards a Platform-Independent Cooperative Human Robot Interaction System: III An Architecture for Learning and Executing Actions and Shared Plans.
IEEE Transactions on Autonomous Mental Development, 4(1), 239–253.
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Lallee, S., Warneken, F., & Dominey, P.
(
2009)
Learning to collaborate by observation. In Epirob, Venice, Italy.
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Lauria, S., Bugmann, G., Kyriacou, T., & Klein, E.
(
2002)
Mobile robot programming using natural language.
Robotics and Autonomous Systems, 381(3–4), 171–181.
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Lévi-Strauss, C.
(
1979)
Myth and Meaning. Routledge.
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Martinez, D., Alenya, G., & Torras, C.
(
2015)
Relational Reinforcement Learning with Guided Demonstrations.
Artificial Intelligence, (Corrected Proof, 19 February 2015).
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
McDermott, D., Malik, G., Adele, H., Craig, K., Ashwin, R., Manuela, V., … David, W.
(
1998)
PDDL – The Planning Domain Definition Language. In
tech. report CVC TR–98-003/DCS TR–1165, Yale Center for Computational Vision and Control, Yale Univ., New Haven, Conn..
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
McGuire, P., Fritsch, J., Steil, J. J., Rothling, F., Fink, G. A., Wachsmuth, S., … Ritter, H.
(
2002)
Multi-modal human-machine communication for instructing robot grasping tasks. In IEEE/RSJ International Conference on Intelligent Robots and System (Vol. 21, pp. 1082–1088). IEEE.
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Metta, G., Sandini, G., Vernon, D., Natale, L., & Nori, F.
(
2008)
The iCub humanoid robot. In
Proceedings of the 8th Workshop on Performance Metrics for Intelligent Systems – PerMIS ’08 (p. 50). New York, New York, USA: ACM Press.
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Mirza, N. A., Nehaniv, C. L., Dautenhahn, K., & Boekhorst, R.
(
2008)
Developing social action capabilities in a humanoid robot using an interaction history architecture. In Humanoids 2008 – 8th IEEE-RAS International Conference on Humanoid Robots (pp. 609–616). IEEE.
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Ognibene, D., Chinellato, E., Sarabia, M., & Demiris, Y.
(
2013)
Contextual action recognition and target localization with an active allocation of attention on a humanoid robot.
Bioinspiration & Biomimetics, 8(1), 035002.
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Pasula, H. M., Zettlemoyer, L. S., & Kaelbling, L. P.
(
2007)
Learning symbolic models of stochastic domains.
Journal of Artificial Intelligence Research, 291, 309–352.
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Payne, J. D., & Nadel, L.
(
2004)
Sleep, dreams, and memory consolidation: the role of the stress hormone cortisol. Learning & Memory (Cold Spring Harbor, N.Y.), 11(1), 671–8.
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Petit, M., Lallee, S., Boucher, J.-D., Pointeau, G., Cheminade, P., Ognibene, D., … Dominey, P. F.
(
2013)
The Coordinating Role of Language in Real-Time Multimodal Learning of Cooperative Tasks.
IEEE Transactions on Autonomous Mental Development, 5(1), 3–17.
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Pointeau, G., Petit, M., & Dominey, P. F.
(
2013)
Embodied Simulation Based on Autobiographical Memory. In
Biomimetic and Biohybrid Systems (Vol. 80641, pp. 240–250). Berlin, Heidelberg: Springer Berlin Heidelberg.
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Pointeau, G., Petit, M., & Dominey, P. F.
(
2013)
Robot Learning Rules of Games by Extraction of Intrinsic Properties. In ACHI 2013, The Sixth International Conference on Advances in Computer-Human Interactions (pp. 109–116).
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Pointeau, G., Petit, M., & Dominey, P. F.
(
2014)
Successive Developmental Levels of Autobiographical Memory for Learning Through Social Interaction.
IEEE Transactions on Autonomous Mental Development, 6(1), 200–212.
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Rader, N. de V., & Zukow-Goldring, P.
(
2012)
Caregivers’ gestures direct infant attention during early word learning: the importance of dynamic synchrony.
Language Sciences, 34(1), 559–568.
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Ross, S., & Pineau, J.
(
2008)
Online Planning Algorithms for POMDPs.
The Journal of Artificial Intelligence Research, 32(1), 663–704.
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Roy, D. K.
(
2002a)
Learning visually grounded words and syntax for a scene description task.
Computer Speech & Language, 161(3–4), 353–385.
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Roy, D. K.
(
2002b)
Learning words from sights and sounds: a computational model.
Cognitive Science, 26(1), 113–146.
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Siskind, J. M.
(
1996)
A computational study of cross-situational techniques for learning word-to-meaning mappings.
Cognition, 611(1–2), 39–91.
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Sutton, S., Ronald A., C., & Jacques De Villiers, Johan Schalkwyk, Pieter JE Vermeulen, Michael W. Macon, Y. Y. et al.
(
1998)
Universal speech tools: the CSLU toolkit. In
ICSLP, vol. 981 (pp. 3221–3224).
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Takács, B., & Demiris, Y.
(
2008)
Balancing Spectral Clustering for Segmenting Spatio-temporal Observations of Multi-agent Systems. In 2008 Eighth IEEE International Conference on Data Mining (pp. 580–587). IEEE.
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Wang, X.
(
1994)
Learning Planning Operators by Observation and Practice. In Proceedings of the Second International Conference on AI Planning Systems, AIPS-94 (pp. 335–340). Chicago, IL.
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Welke, K., Kaiser, P., Kozlov, A., Adermann, N., Asfour, T., Lewis, M., & Steedman, M.
(
2013)
Grounded Spatial Symbols for Task Planning Based on Experience. In 13th International Conference on Humanoid Robots (Humanoids). IEEE/RAS.
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Wood, R., Baxter, P., & and Belpaeme, T.
(
2011)
A review of long-term memory in natural and synthetic systems.
Adaptive Behavior, 20(1), 81–103.
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Zacks, J. M., Speer, N. K., Swallow, K. M., Braver, T. S., & Reynolds, J. R.
(
2007)
Event perception: A mind-brain perspective.
Psychological bulletin2, 1331, 273.
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Cited by
Cited by 3 other publications
Kumar, Suresh, Patricia Shaw, Alexandros Giagkos, Raphäel Braud, Mark Lee & Qiang Shen
2018.
Developing Hierarchical Schemas and Building Schema Chains Through Practice Play Behavior.
Frontiers in Neurorobotics 12
![DOI logo](//benjamins.com/logos/doi-logo.svg)
Pointeau, Gregoire & Peter Ford Dominey
2017.
The Role of Autobiographical Memory in the Development of a Robot Self.
Frontiers in Neurorobotics 11
![DOI logo](//benjamins.com/logos/doi-logo.svg)
Sarthou, Guillaume, Guilhem Buisan, Aurelie Clodic & Rachid Alami
2021.
2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS),
► pp. 1879 ff.
![DOI logo](//benjamins.com/logos/doi-logo.svg)
This list is based on CrossRef data as of 1 june 2024. Please note that it may not be complete. Sources presented here have been supplied by the respective publishers.
Any errors therein should be reported to them.