An epistemic logic for formalizing group dynamics of agents
In the multi-agent setting, it is relevant to model group dynamics of agents, and logic has proved a good tool to do so. We propose an epistemic logic, L-DINF-E, that allows one to formalize what are the beliefs formed by a group of agents, where several groups exist and agents can pass from a group to another one. We introduce a new modality which allows an agent to reason about the beliefs of other agents. This allows us to model aspects of the “Theory of Mind”, understood as the set of social-cognitive skills involving the ability to attribute and reason about mental states, desires, beliefs, and knowledge of agents. In this paper, we present the logic L-DINF-E and illustrate how it can be used to solve “false-belief tasks”, i.e., tests in which an agent should understand that some other agent may develop, under some circumstances, false beliefs.
Article outline
- Introduction
- Background on agents and verification, and on the DALI Language
- Logical framework
- False-beliefs: The Sally-Anne task
- Future directions and conclusion
- Acknowledgements
- Notes
-
References
References (83)
References
Adam, C., Herzig, A., & Longin, D. (2009). A logical formalization of the OCC theory of emotions. Synth.,
168
(2), 201–248.
Aielli, F., Ancona, D., Caianiello, P., Costantini, S., De Gasperis, G., Di Marco, A., … Mascardi, V. (2016). FRIENDLY & KIND with your health: Human-friendly knowledge-intensive dynamic systems for the e-health domain. In J. Bajo et al. (Eds.), Highlights of practical applications of scalable multi-agent systems. The PAAMS collection – international workshops of PAAMS 2016, proceedings (Vol. 616 of Communications in Computer and Information Science, pp. 15–26). Springer.
Alberti, M., Chesani, F., Gavanelli, M., Lamma, E., Mello, P., & Torroni, P. (2008). Verifiable agent interaction in abductive logic programming: The SCIFF framework. ACM Trans. Comput. Log.,
9
(4), 291:1–29:431.
Amir, E., Andreson, M. L., & Chaudri, V. K. (2007). Report on DARPA workshop on self aware computer systems. Tech. Rep., SRI International Menlo Park, USA. (Full text: [URL])
Balaji, P. G., & Srinivasan, D. (2010). An introduction to multi-agent systems. In D. Srinivasan & L. C. Jain (Eds.), Innovations in MASs and applications (Vol. 310 of Studies in Computational Intelligence). Springer.
Balbiani, P., Fernández-Duque, D., & Lorini, E. (2016). A logical theory of belief dynamics for resource-bounded agents. In Proceedings of AAMAS 2016 (pp. 644–652). ACM.
Balbiani, P., Fernandez-Duque, D., & Lorini, E. (2019). The dynamics of epistemic attitudes in resource-bounded agents. Studia Logica,
107
(3), 457–488.
Baron-Cohen, S., Leslie, A. M., & Frith, U. (1985). Does the autistic child have a “Theory of Mind” ? Cognition,
21
(1), 37–46.
Bolander, T., Dissing, L., & Herrmann, N. (2021). DEL-based epistemic planning for human-robot collaboration: Theory and implementation. In M. Bienvenu, G. Lakemeyer, & E. Erdem (Eds.), Proceedings of the 18th international conference on principles of knowledge representation and reasoning, KR 2021, online event, november 3–12, 2021 (pp. 120–129).
Bordini, R. H., Braubach, L., Dastani, M., El Fallah-Seghrouchni, A., Gómez-Sanz, J. J., Leite, J., … Ricci, A. (2006). A survey of programming languages and platforms for multi-agent systems. Informatica (Slovenia),
30
(1), 33–44.
Bordini, R. H., Fisher, M., Visser, W., & Wooldridge, M. (2006). Verifying multi-agent programs by model checking. Autonomous Agents and Multi-Agent Systems,
12
(2), 239–256.
Bordini, R. H., & Hübner, J. F. (2005). BDI agent programming in AgentSpeak using Jason (tutorial paper). In F. Toni & P. Torroni (Eds.), Computational logic in multi-agent systems, 6th international workshop, CLIMA VI, revised selected and invited papers (Vol. 3900 of Lecture Notes in Computer Science, pp. 143–164). Springer.
Bosse, T., Memon, Z. A., & Treur, J. (2011). A recursive BDI agent model for theory of mind and its applications. Applied Artificial Intelligence,
25
(1), 1–44.
Bösser, T. (2001). Autonomous agents. In N. J. Smelser & P. B. Baltes (Eds.), International encyclopedia of the social & behavioral sciences. Science Direct, Elsevier.
Calegari, R., Ciatto, G., Mascardi, V., & Omicini, A. (2021). Logic-based technologies for multi-agent systems: a systematic literature review. Auton. Agents Multi Agent Syst.,
35
(1), 1.
Carlsson, M., & Mildner, P. (2012). SICStus Prolog – the first 25 years. Theory Pract. Log. Program.,
12
(1–2), 35–66.
Clark, K., & Robinson, P. (2021). Qulog/TeleoR home page. ([URL])
Costantini, S. (2011). Defining and maintaining agent’s experience in logical agents. In Proceedings of LANMR 2011, seventh Latin-American workshop on non-monotonic reasoning (Vol. 804 of CEUR Workshop Proceedings, p. 151–165). [URL]
Costantini, S. (2015). ACE: a flexible environment for complex event processing in logical agents. In M. Baldoni, L. Baresi, & M. Dastani (Eds.), Engineering multi-agent systems, third intl. works., EMAS 2015, revised selected papers (Vol. 9318 of Lecture Notes in Computer Science). Springer.
Costantini, S. (2022). Ensuring trustworthy and ethical behaviour in intelligent logical agents. J. Log. Comput.,
32
(2), 443–478.
Costantini, S., & De Gasperis, G. (2018). Flexible goal-directed agents’ behavior via DALI MASs and ASP modules. In 2018 AAAI spring symposia. AAAI Press.
Costantini, S., De Gasperis, G., Dyoub, A., & Pitoni, V. (2018). Trustworthiness and safety for intelligent ethical logical agents via interval temporal logic and runtime self-checking. In AAAI spring symposia, Stanford university. AAAI Press.
Costantini, S., De Gasperis, G., & Migliarini, P. (2019). Multi-agent system engineering for emphatic human-robot interaction. In 2019 IEEE second international conference on artificial intelligence and knowledge engineering (AIKE) (pp. 36–42).
Costantini, S., De Gasperis, G., & Nazzicone, G. (2017). DALI for cognitive robotics: Principles and prototype implementation. In Y. Lierler & W. Taha (Eds.), Practical aspects of declarative languages – 19th international symposium, proceedings (Vol. 10137 of Lecture Notes in Computer Science, pp. 152–162). Springer.
Costantini, S., De Gasperis, G., Pitoni, V., & Salutari, A. (2017). DALI: A multi agent system framework for the web, cognitive robotic and complex event processing. In D. Monica, A. Murano, S. Rubin, & L. Sauro (Eds.), Proceedings of ICTCS’17 and CILC’17 (Vol. 1949 of CEUR Workshop Proceedings, pp. 286–300). [URL]
Costantini, S., De Lauretis, L., Ferri, C., Giancola, J., & Persia, F. (2021). A smart health assistant via DALI logical agents. In S. Monica & F. Bergenti (Eds.), Proceedings of the 36th Italian conference on computational logic, 2021 (Vol. 3002 of CEUR Workshop Proceedings, pp. 173–187). [URL]
Costantini, S., De Lauretis, L., & Persia, F. (2022a). Intelligent agents and complex event processing to enhance patient monitoring. In M. Alviano & A. Pieris (Eds.), Proceedings Datalog 2.0 2022, co-located with LPNMR 2022. [URL]
Costantini, S., De Lauretis, L., & Persia, F. (2022b). An intelligent ecosystem to improve patient monitoring using wearables and artificial intelligence. In R. Calegari & G. Ciatto (Eds.), Proceedings of CILC 2022, 37th Italian conference on computational logic. [URL]
Costantini, S., Dell’Acqua, P., & Pereira, L. M. (2011). Conditional learning of rules and plans by knowledge exchange in logical agents. In N. Bassiliades, G. Governatori, & A. Paschke (Eds.), Rule-based reasoning, programming, and applications – proceedings of 5th international symposium, RuleML 2011 (Vol. 6826 of Lecture Notes in Computer Science, pp. 250–265). Springer.
Costantini, S., Dyoub, A., & Pitoni, V. (2018). Towards humanized ethical intelligent agents: the role of reflection and introspection. In P. Felli & M. Montali (Eds.), Proceedings of the 33rd Italian conference on computational logic (Vol. 2214 of CEUR Workshop Proceedings, pp. 82–96). [URL]. Retrieved from [URL]
Costantini, S., Formisano, A., & Pitoni, V. (2021a). An epistemic logic for modular development of multi-agent systems. In N. Alechina, M. Baldoni, & B. Logan (Eds.), Engineering multi-agent systems – 9th international workshop, EMAS 2021, revised selected papers (Vol. 13190 of Lecture Notes in Computer Science, pp. 72–91). Springer.
Costantini, S., Formisano, A., & Pitoni, V. (2021b). An epistemic logic for multi-agent systems with budget and costs. In W. Faber, G. Friedrich, M. Gebser, & M. Morak (Eds.), Logics in artificial intelligence – 17th European conference, JELIA 2021, proceedings (Vol. 12678 of Lecture Notes in Computer Science, pp. 101–115). Springer.
Costantini, S., Formisano, A., & Pitoni, V. (2021c). A logic of inferable in multi-agent systems with budget and costs. In F. Dignum, A. Lomuscio, U. Endriss, & A. Nowé (Eds.), AAMAS ’21: 20th international conference on autonomous agents and multiagent systems, virtual event, UK, 2021 (pp. 1483–1485). ACM.
Costantini, S., Formisano, A., & Pitoni, V. (2022a). Cooperation among groups of agents in the epistemic logic L-DINF. In G. Governatori & A.-Y. Turhan (Eds.), Rules and reasoning – 6th international joint conference on rules and reasoning, Rule ML+RR 2022, Berlin, Germany, 2022, proceedings (Vol. 13752 of Lecture Notes in Computer Science). Springer.
Costantini, S., Formisano, A., & Pitoni, V. (2022b). Modelling agents roles in the epistemic logic L-DINF. In O. Arieli, G. Casini, & L. Giordano (Eds.), Proceedings of the 20th international workshop on non-monotonic reasoning, NMR 2022, part of the federated logic conference FLoC 2022, Haifa, Israel, 2022 (Vol. 3197 of CEUR Workshop Proceedings, pp. 70–79). [URL]
Costantini, S., Formisano, A., & Pitoni, V. (2022c). Temporalizing epistemic logic L-DINF. In R. Calegari, G. Ciatto, & A. Omicini (Eds.), Proceedings of the 37th Italian conference on computational logic, Bologna, Italy, 2022 (Vol. 3204 of CEUR Workshop Proceedings, pp. 119–133). [URL]
Costantini, S., & Pitoni, V. (2020). Towards a logic of “inferable” for self-aware transparent logical agents. In C. Musto, D. Magazzeni, S. Ruggieri, & G. Semeraro (Eds.), Proceedings of XAI.it@AIxIA 2020 (Vol. 2742 of CEUR Workshop Proceedings, pp. 68–79). [URL]. Retrieved from [URL]
Costantini, S., & Tocchio, A. (2002). A logic programming language for multi-agent systems. In S. Flesca, S. Greco, N. Leone, & G. Ianni (Eds.), Logics in artificial intelligence, European conference, JELIA 2002, proceedings (Vol. 2424 of Lecture Notes in Computer Science, pp. 1–13). Springer.
Costantini, S., & Tocchio, A. (2004). The DALI logic programming agent-oriented language. In J. J. Alferes & J. A. Leite (Eds.), Logics in artificial intelligence, 9th European conference, JELIA 2004, proceedings (Vol. 3229 of Lecture Notes in Computer Science, pp. 685–688). Springer.
Costantini, S., & Tocchio, A. (2005). About declarative semantics of logic-based agent languages. In M. Baldoni, U. Endriss, A. Omicini, & P. Torroni (Eds.), Declarative agent languages and technologies III, third international workshop, DALT 2005, selected and revised papers (Vol. 3904 of Lecture Notes in Computer Science, pp. 106–123). Springer.
Costantini, S., Tocchio, A., & Verticchio, A. (2005). Communication and trust in the DALI logic programming agent-oriented language. Intelligenza Artificiale,
2
(1), 39–46. (Journal of the Italian Association AI*IA)
Dastani, M., van Riemsdijk, M. B., & Meyer, J. C. (2005). Programming multi-agent systems in 3APL. In Multi-agent programming (Vol. 15 of Multiagent Systems Artificial Societies and Simulated Organizations, pp. 39–67). Springer.
De Gasperis, G., Costantini, S., & Nazzicone, G. (2014, July). DALI multi agent systems framework. DALI GitHub Software Repository. (DALI: [URL])
de Weerd, H., Verbrugge, R., & Verheij, B. (2022). Higher-order theory of mind is especially useful in unpredictable negotiations. Auton. Agents Multi Agent Syst.,
36
(1), 30.
Dennis, L. A. (2018). The MCAPL framework including the agent infrastructure layer an agent Java pathfinder. J. of Open Source Software,
3
(24), 617.
Dennis, L. A., Bentzen, M. M., Lindner, F., & Fisher, M. (2021). Verifiable machine ethics in changing contexts. In Thirty-fifth AAAI conference on artificial intelligence, AAAI 2021, thirty-third conference on innovative applications of artificial intelligence, IAAI 2021, the eleventh symposium on educational advances in artificial intelligence, EAAI 2021 (pp. 11470–11478). AAAI Press.
Dennis, L. A., Fisher, M., Lincoln, N., Lisitsa, A., & Veres, S. M. (2016). Practical verification of decision-making in agent-based autonomous systems. Autom. Softw. Eng.,
23
(3), 305–359.
Dissing, L., & Bolander, T. (2020). Implementing theory of mind on a robot using dynamic epistemic logic. In C. Bessiere (Ed.), Proceedings of IJCAI 2020 (pp. 1615–1621). [URL].
Dorri, A., Kanhere, S. S., & Jurdak, R. (2018). Multi-agent systems: A survey. IEEE Access,
6
1, 28573–28593.
Duží, M., & Menšík, M. (2017). Logic of inferable knowledge. In Frontiers in Artificial Intelligence and Applications, Information Modelling and Knowledge Bases XXVIII, Volume 292. IOS Press.
Dyoub, A., Costantini, S., Letteri, I., & Lisi, F. A. (2021). A logic-based multi-agent system for ethical monitoring and evaluation of dialogues. In A. Formisano et al. (Eds.), Proceedings of the 37th international conference on logic programming (technical communications) (Vol. 345 of Electronic Proceedings in Theoretical Computer Science, pp. 182–188).
Dyoub, A., Costantini, S., & Lisi, F. A. (2019). Towards ethical machines via logic programming. In B. Bogaerts et al. (Eds.), Proceedings of the 35th international conference on logic programming (technical communications) (Vol. 306 of Electronic Proceedings in Theoretical Computer Science, pp. 333–339).
Ferrando, A., Dennis, L. A., Ancona, D., Fisher, M., & Mascardi, V. (2018). Verifying and validating autonomous systems: Towards an integrated approach. In C. Colombo & M. Leucker (Eds.), Runtime verification, 18th international conference, RV 2018, proceedings (Vol. 11237 of Lecture Notes in Computer Science, pp. 263–281). Springer.
Ferrando, A., Winikoff, M., Cranefield, S., Dignum, F., & Mascardi, V. (2019). On enactability of agent interaction protocols: Towards a unified approach. In L. A. Dennis, R. H. Bordini, & Y. Lespérance (Eds.), Engineering multi-agent systems – 7th international workshop, EMAS 2019, revised selected papers (Vol. 12058 of Lecture Notes in Computer Science, pp. 43–64). Springer.
Ferrario, A., & Loi, M. (2022). How explainability contributes to trust in AI. In Facct ’22: 2022 ACM conference on fairness, accountability, and transparency (pp. 1457–1466). ACM.
Fisher, M., Bordini, R. H., Hirsch, B., & Torroni, P. (2007). Computational logics and agents: a road map of current technologies and future trends. Computational Intelligence Journal,
23
(1), 61–91.
Garro, A., Mühlhäuser, M., Tundis, A., Baldoni, M., Baroglio, C., Bergenti, F., & Torroni, P. (2019). Intelligent agents: Multi-agent systems. In S. Ranganathan, M. Gribskov, K. Nakai, & C. Schonbach (Eds.), Encyclopedia of bioin formatics and computational biology – Vol. 1 (pp. 315–320). Elsevier.
Goldman, A. I. (2012). Theory of mind. In E. Margolis, R. Samuels, & S. P. Stich (Eds.), The Oxford handbook of philosophy of cognitive science (Vol. 11). Oxford University Press.
Hindriks, K. V., van der Hoek, W., & Meyer, J. C. (2012). GOAL agents instantiate intention logic. In Logic programs, norms and action (Vol. 7360 of Lecture Notes in Computer Science, pp. 196–219). Springer.
Holzmann, G. J. (1991). Design and validation of computer protocols. Prentice Hall Intl.: Hemel Hempstead, England.
Jones, A. V., & Lomuscio, A. (2010). Distributed BDD-based BMC for the verification of multi-agent systems. In W. van der Hoek, G. A. Kaminka, Y. Lespérance, M. Luck, & S. Sen (Eds.), 9th international conference on autonomous agents and multiagent systems (AAMAS 2010) (pp. 675–682). IFAAMAS.
Kacprzak, M., Lomuscio, A., & Penczek, W. (2004). Verification of multiagent systems via unbounded model checking. In Proceedings of the third int. joint conf. on autonomous agents and multiagent systems, AAMAS 04 (p. 638–645). ACM Press.
Kong, J., & Lomuscio, A. (2017). Symbolic model checking multi-agent systems against CTL*K specifications. In K. Larson, M. Winikoff, S. Das, & E. H. Durfee (Eds.), Proceedings of the 16th conference on autonomous agents and multiagent systems, AAMAS 2017 (pp. 114–122). ACM.
Lloyd, J. W. (1987). Foundations of logic programming, second edition. Berlin: Springer.
Lomuscio, A., Lasica, T., & Penczek, W. (2002). Bounded model checking for interpreted systems: Preliminary experimental results. In M. G. Hinchey, J. L. Rash, W. Truszkowski, C. A. Rouff, & D. F. Gordon-Spears (Eds.), Formal approaches to agent-based systems, 2nd international workshop, FAABS 2002, revised papers (Vol. 2699 of Lecture Notes in Computer Science, pp. 115–125). Springer.
Lomuscio, A., Qu, H., & Raimondi, F. (2017). MCMAS: an open-source model checker for the verification of multi-agent systems. Int. J. Softw. Tools Technol. Transf.,
19
(1), 9–30.
Martinich, A. P. (2009). The philosophy of language. Oxford University Press. (International Fifth Edition)
McMillan, K. L. (1993). Symbolic model checking. Kluwer Academic Publishers.
Panisson, A. R., Sarkadi, S., McBurney, P., Parsons, S., & Bordini, R. H. (2018). On the formal semantics of theory of mind in agent communication. In M. Lujak (Ed.), Agreement technologies – 6th international conference, AT 2018, revised selected papers (Vol. 11327 of Lecture Notes in Computer Science, pp. 18–32). Springer.
Rao, A. S., & Georgeff, M. P. (1991). Modeling rational agents within a BDI-architecture. In J. F. Allen, R. Fikes, & E. Sandewall (Eds.), Proceedings of the 2nd international conference on principles of knowledge representation and reasoning (KR 91) (pp. 473–484). Morgan Kaufmann.
Reisenzein, R., Hudlicka, E., Dastani, M., Gratch, J., Hindriks, K. V., Lorini, E., & Meyer, J. C. (2013). Computational modeling of emotion: Toward improving the inter- and intradisciplinary exchange. IEEE Trans. Affect. Comput.,
4
(3), 246–266.
Rozier, K. Y. (2011). Linear temporal logic symbolic model checking. Comput. Sci. Rev.,
5
(2), 163–203.
Rozier, K. Y. (2016). Specification: The biggest bottleneck in formal methods and autonomy. In S. Blazy & M. Chechik (Eds.), Verified software. theories, tools, and experiments – 8th international conference, VSTTE 2016, revised selected papers (Vol. 9971 of Lecture Notes in Computer Science, pp. 8–26).
Rozier, K. Y., & Vardi, M. Y. (2010). LTL satisfiability checking. Int. J. Softw. Tools Technol. Transf.,
12
(2), 123–137.
Sarkadi, S., Panisson, A. R., Bordini, R. H., McBurney, P., & Parsons, S. (2018). Towards an approach for modelling uncertain theory of mind in multi-agent systems. In M. Lujak (Ed.), Agreement technologies – 6th international conference, AT 2018, revised selected papers (Vol. 11327 of Lecture Notes in Computer Science, pp. 3–17). Springer.
Stone, P., & Veloso, M. M. (2000). Multiagent systems: A survey from a machine learning perspective. Auton. Robots,
8
(3), 345–383.
Tørresen, J., Plessl, C., & Yao, X. (2015). Self-aware and self-expressive systems. IEEE Computer,
48
(7), 18–20.
van Ditmarsch, H., Halpern, J. Y., van der Hoek, W., & Kooi, B. (2015). Handbook of epistemic logic. College Publications.
Vardi, M. Y. (2001). Branching vs. linear time: Final showdown. In Proceedings of the 2001 conf. on tools and algorithms for the construction and analysis of systems, TACAS 2001 (p. 1–22). Springer-Verlag.
Veltman, K., de Weerd, H., & Verbrugge, R. (2019). Training the use of theory of mind using artificial agents. J. Multimodal User Interfaces,
13
(1), 3–18.
Weyhrauch, R. W. (1980). Prolegomena to a theory of mechanized formal reasoning. Artif. Intell.,
13
(1–2), 133–170.
Yakouda, M., Abbel, W., Corneille, K. V., & Sinclair, N. D. (2020). Multi-agent system: A two-level BDI model integrating theory of mind. International Journal of Engineering Research & Technology (IJERT),
9
(7). Retrieved from [URL]
Zhao, S., Wang, S., Soleymani, M., Joshi, D., & Ji, Q. (2019). Affective computing for large-scale heterogeneous multimedia data: A survey. ACM Transactions on Multimedia Computing, Communications, and Applications,
15
(
3s
), 1–32.
Cited by (2)
Cited by two other publications
Costantini, Stefania, Andrea Formisano & Valentina Pitoni
2024.
Epistemic Logic and Theory of Mind for Modelling Group Dynamics of Criminal Organizations. In
Computational Intelligence and Mathematics for Tackling Complex Problems 5 [
Studies in Computational Intelligence, 1127],
► pp. 123 ff.
Roesler, Oliver, Elahe Bagheri, Amir Aly, Silvia Rossi & Rachid Alami
This list is based on CrossRef data as of 4 july 2024. Please note that it may not be complete. Sources presented here have been supplied by the respective publishers.
Any errors therein should be reported to them.