Cover not available

Article In: Pragmatics and Society: Online-First Articles

Pragmatics-zombies

Trust and communication in the age of artificial intelligence

 | University of Haifa
ORCiD logo with link | University of Haifa
This content is being prepared for publication; it may be subject to changes.
Abstract
This paper examines theoretical aspects of human-AI communication, arguing that current AI systems, or ‘pragmatics-zombies,’ can mimic human-like pragmatic competence without genuine understanding. While these systems are increasingly adept at natural language interaction, their lack of subjective experience, intentionality, and embodied cognition creates a fundamental trust deficit, particularly in high-stakes scenarios. We argue that forcing human-to-human pragmatic frameworks onto these non-human entities is suboptimal, leading to issues in efficiency, robustness, and transparency.
To address this, we introduce the concept of ‘Abstract Pragmatics,’ a general framework that allows for multiple, distinct pragmatic systems. We propose that instead of pursuing flawless mimicry of human communication, research should focus on developing novel, alternative pragmatic systems specifically designed for human-AI interaction. This approach would prioritize clarity, verifiability, and trustworthiness, fostering a more effective and reliable collaboration between humans and artificial intelligence by acknowledging and adapting to the unique nature of the machine.
Article outline
  • 1.Introduction
  • 2.Can we trust the pragmatics-zombie?
  • 3.Optimization: A delicate balance of constraints
  • 4.Abstract Pragmatics and its multiple realizations
  • 5.An outline of different pragmatics
  • 6.Concluding remarks
  • Notes
  • Author queries
  • References
References (90)
References
Allen, James F., and C. Raymond Perrault. 1980. “Analyzing Intention in Utterances.” Artificial Intelligence 15 (3): 143–178. Google Scholar logo with link to Google Scholar
Amodei, Dario, Chris Olah, Jacob Steinhardt, Paul Christiano, John Schulman, and Dan Mané. 2016. “Concrete Problems in AI Safety.” Google Scholar logo with link to Google Scholar
Avramides, Anita. 2023. “Other minds.” In The Stanford Encyclopedia of Philosophy (Winter 2023 ed.), ed. by Edward N. Zalta, and Uri Nodelman. [URL]
Baltrušaitis, Tadas, Chaitanya Ahuja, and Louis-Philippe Morency. 2018. “Multimodal Machine Learning: A Survey and Taxonomy.” IEEE Transactions on Pattern Analysis and Machine Intelligence 41 (2): 423–443. Google Scholar logo with link to Google Scholar
Bender, Emily M., Timnit Gebru, Angelina McMillan-Major, and Shmargaret Shmitchell. 2021. “On the Dangers of Stochastic Parrots: Can Language Models Be Too Big?” In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency (FAccT ’21), 610–623. New York: Association for Computing Machinery. Google Scholar logo with link to Google Scholar
Bickmore, Timothy, and Justine Cassell. 2001. “Relational Agents: A Model and Implementation of Building User Trust.” In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ’01), 396–403. New York: Association for Computing Machinery. Google Scholar logo with link to Google Scholar
Bisk, Yonatan, Ari Holtzman, Jesse Thomason, et al. 2020. “Experience Grounds Language.” In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), 8718–8735. Online: Association for Computational Linguistics. Google Scholar logo with link to Google Scholar
Block, Ned. 1980a. “Troubles with Functionalism.” In Readings in Philosophy of Psychology, vol. 11, ed. by Ned Block, 268–305. Cambridge, Mass.: Harvard University Press.Google Scholar logo with link to Google Scholar
. 1980b. “Are Absent Qualia Impossible?” Philosophical Review 89 (2): 257–274. Google Scholar logo with link to Google Scholar
. 1981. “Psychologism and Behaviorism.” Philosophical Review 90 (1): 5–43. Google Scholar logo with link to Google Scholar
Brooks, Rodney A. 1991. “Intelligence without Representation.” Artificial Intelligence 47 (1–3): 139–159. Google Scholar logo with link to Google Scholar
Brown, Tom B., Benjamin Mann, Nick Ryder, et al. 2020. “Language Models Are Few-Shot Learners.” In Advances in Neural Information Processing Systems 331: 1877–1901. Red Hook, NY: Curran Associates. Google Scholar logo with link to Google Scholar
Brunke, Lukas, Melissa Greeff, Adam W. Hall, et al. 2022. “Safe Learning in Robotics: From Learning-Based Control to Safe Reinforcement Learning.” Annual Review of Control, Robotics, and Autonomous Systems 51: 411–444. Google Scholar logo with link to Google Scholar
Burtsev, Mikhail, Martin Reeves, and Adam Job. 2023. “The Working Limitations of Large Language Models.” MIT Sloan Management Review, November 30, 2023. [URL]
Cassell, Justine, Joseph Sullivan, Scott Prevost, and Elizabeth F. Churchill (eds). 2000. Embodied Conversational Agents. Cambridge, Mass.: MIT Press. Google Scholar logo with link to Google Scholar
Chalmers, David J. 1996. The Conscious Mind: In Search of a Fundamental Theory. Oxford: Oxford University Press.Google Scholar logo with link to Google Scholar
Chiang, Ted. 2002. “Story of Your Life.” In Stories of Your Life and Others, 91–145. New York: Tor Books.Google Scholar logo with link to Google Scholar
Ciarrochi, Joseph, Joseph P. Forgas, and John D. Mayer (eds). 2001. Emotional Intelligence in Everyday Life: A Scientific Inquiry. Philadelphia: Psychology Press. Google Scholar logo with link to Google Scholar
Clark, Andy. 1997. Being There: Putting Brain, Body, and World Together Again. Cambridge, Mass.: MIT Press.Google Scholar logo with link to Google Scholar
Clark, Herbert H., and Edward F. Schaefer. 1989. “Contributing to Discourse.” Cognitive Science 13 (2): 259–294. Google Scholar logo with link to Google Scholar
Cong, Yan. 2024. “Manner Implicatures in Large Language Models.” Scientific Reports 141: article 29113. Google Scholar logo with link to Google Scholar
Dahiya, Ravinder S., Giorgio Metta, Maurizio Valle, and Giulio Sandini. 2010. “Tactile Sensing: From Humans to Humanoids.” IEEE Transactions on Robotics 26 (1): 1–20. Google Scholar logo with link to Google Scholar
Doshi-Velez, Finale, and Been Kim. 2017. “Towards a Rigorous Science of Interpretable Machine Learning.” In Proceedings of the 2017 ICML Workshop on Human Interpretability in Machine Learning (WHI 2017). Sydney. Google Scholar logo with link to Google Scholar
Dragan, Anca D., Kenton C. T. Lee, and Siddhartha S. Srinivasa. 2013. “Legibility and Predictability of Robot Motion.” In Proceedings of the 2013 8th ACM/IEEE International Conference on Human-Robot Interaction (HRI), 301–308. New York: Association for Computing Machinery. Google Scholar logo with link to Google Scholar
Falguera, José L., Concha Martínez-Vidal, and Gideon Rosen. 2022. “Abstract objects.” In The Stanford Encyclopedia of Philosophy (Summer 2022 ed.), ed. by Edward N. Zalta. [URL]
Firt, Erez. 2025. “What Makes Full Artificial Agents Morally Different.” AI & Society 40 (1): 175–184. Google Scholar logo with link to Google Scholar
Fischer, Kerstin. 2011. “How People Talk with Robots: Designing Dialog to Reduce User Uncertainty.” AI Magazine 32 (4): 31–38. Google Scholar logo with link to Google Scholar
Garcez, Artur d’Avila, and Luís C. Lamb. 2023. “Neurosymbolic AI: The 3rd Wave.” Artificial Intelligence Review 56 (11): 12387–12406. Google Scholar logo with link to Google Scholar
Glikson, Ella, and Anita Williams Woolley. 2020. “Human Trust in Artificial Intelligence: Review of Empirical Research.” Academy of Management Annals 14 (2): 627–660. Google Scholar logo with link to Google Scholar
Goddard, Kate, Abdul Roudsari, and Jeremy C. Wyatt. 2012. “Automation Bias: A Systematic Review of Frequency, Effect Mediators, and Mitigations.” Journal of the American Medical Informatics Association 19 (1): 121–127. Google Scholar logo with link to Google Scholar
Goleman, Daniel. 2004. Emotional Intelligence: Why It Can Matter More Than IQ. London: Bloomsbury.Google Scholar logo with link to Google Scholar
Grice, H. Paul. 1975. “Logic and Conversation.” In Syntax and Semantics, Vol. 3: Speech Acts, ed. by Peter Cole, and Jerry L. Morgan, 41–58. New York: Academic Press. Google Scholar logo with link to Google Scholar
Grosz, Barbara J., and Candace L. Sidner. 1986. “Attention, Intentions, and the Structure of Discourse.” Computational Linguistics 12 (3): 175–204.Google Scholar logo with link to Google Scholar
Hancock, Jeffrey T., Mor Naaman, and Karen Levy. 2020. “AI-Mediated Communication: Definition, Research Agenda, and Ethical Considerations.” Journal of Computer-Mediated Communication 25 (1): 89–100. Google Scholar logo with link to Google Scholar
Harnad, Stevan. 1990. “The Symbol Grounding Problem.” Physica D: Nonlinear Phenomena 42 (1–3): 335–346. Google Scholar logo with link to Google Scholar
Ji, Ziwei, Nayeon Lee, Rita Frieske, et al. 2023. “Survey of Hallucination in Natural Language Generation.” ACM Computing Surveys 55 (12): article 248. Google Scholar logo with link to Google Scholar
Jones, Erik, and Jacob Steinhardt. 2022. “Capturing Failures of Large Language Models via Human Cognitive Biases.” In Advances in Neural Information Processing Systems 351, ed. by Sanmi Koyejo et al., 11785–11799. Red Hook, NY: Curran Associates. Google Scholar logo with link to Google Scholar
Jonze, Spike, dir. 2013. Her. Burbank, Calif.: Warner Bros. Pictures.Google Scholar logo with link to Google Scholar
Jurafsky, Daniel, and James H. Martin. 2023. Speech and Language Processing: An Introduction to Natural Language Processing, Computational Linguistics, and Speech Recognition. 3rd ed. draft. [URL]
Kendon, Adam. 2004. Gesture: Visible Action as Utterance. Cambridge: Cambridge University Press. Google Scholar logo with link to Google Scholar
Kirk, Robert. 1974. “Zombies v. Materialists.” Proceedings of the Aristotelian Society, Supplementary Volumes 481: 135–152. Google Scholar logo with link to Google Scholar
Kosinski, Michal. 2024. “Evaluating Large Language Models in Theory of Mind Tasks.” Proceedings of the National Academy of Sciences 121 (45): e2405460121. Google Scholar logo with link to Google Scholar
Lazaridou, Angeliki, Alexander Peysakhovich, and Marco Baroni. 2017. “Multi-Agent Cooperation and the Emergence of (Natural) Language.” Paper presented at the International Conference on Learning Representations (ICLR), Toulon, France.
Leckie, Ann. 2013. Ancillary Justice. New York: Orbit.Google Scholar logo with link to Google Scholar
Lee, John D., and Katrina A. See. 2004. “Trust in Automation: Designing for Appropriate Reliance.” Human Factors 46 (1): 50–80. Google Scholar logo with link to Google Scholar
Lem, Stanisław. (1961) 1970. Solaris. Translated by Joanna Kilmartin, and Steve Cox. New York: Walker.Google Scholar logo with link to Google Scholar
Levinson, Stephen C. 1983. Pragmatics. Cambridge: Cambridge University Press. Google Scholar logo with link to Google Scholar
Lewis, David. 1986. On the Plurality of Worlds. Oxford: Blackwell.Google Scholar logo with link to Google Scholar
Li, Yugang, Baizhou Wu, Yuqi Huang, and Shenghua Luan. 2024. “Developing Trustworthy Artificial Intelligence: Insights from Research on Interpersonal, Human-Automation, and Human-AI Trust.” Frontiers in Psychology 151: 1382693. Google Scholar logo with link to Google Scholar
Lin, Stephanie, Jacob Hilton, and Owain Evans. 2022. “Teaching Models to Express Their Uncertainty in Words.” Transactions on Machine Learning Research. [URL]
Luger, Ewa, and Abigail Sellen. 2016. “Like Having a Really Bad PA: The Gulf between User Expectation and Experience of Conversational Agents.” In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems, 5286–5297. New York: Association for Computing Machinery. Google Scholar logo with link to Google Scholar
MacKenzie, I. Scott. 2024. Human-Computer Interaction: An Empirical Research Perspective. 2nd ed. Cambridge, Mass.: Morgan Kaufmann.Google Scholar logo with link to Google Scholar
Malle, Bertram F., Kerstin Fischer, James E. Young, AJung Moon, and Emily C. Collins. 2020. “Trust and the Discrepancy between Expectations and Actual Capabilities of Social Robots.” In Human-Robot Interaction: Control, Analysis, and Design, ed. by Dan Zhang, and Bin Wei, 1–23. Newcastle upon Tyne: Cambridge Scholars Publishing.Google Scholar logo with link to Google Scholar
Marcus, Gary. 2020. “The Next Decade in AI: Four Steps Towards Robust Artificial Intelligence.” Google Scholar logo with link to Google Scholar
. 2022. “Deep Learning Is Hitting a Wall.” Nautilus, March 10, 2022. [URL]
Marcus, Gary, and Ernest Davis. 2019. Rebooting AI: Building Artificial Intelligence We Can Trust. New York: Pantheon Books.Google Scholar logo with link to Google Scholar
Mayer, Roger C., James H. Davis, and F. David Schoorman. 1995. “An Integrative Model of Organizational Trust.” Academy of Management Review 20 (3): 709–734. Google Scholar logo with link to Google Scholar
Mielke, Sabrina J., Arthur Szlam, Emily Dinan, and Y-Lan Boureau. 2022. “Reducing Conversational Agents’ Overconfidence through Linguistic Calibration.” Transactions of the Association for Computational Linguistics 101: 857–872. Google Scholar logo with link to Google Scholar
Morris, Charles W. 1938. Foundations of the Theory of Signs. International Encyclopedia of Unified Science 1 (2). Chicago: University of Chicago Press.Google Scholar logo with link to Google Scholar
(1946) 1955. Signs, Language, and Behavior. New York: George Braziller.Google Scholar logo with link to Google Scholar
Nagel, Thomas. 1974. “What Is It Like to Be a Bat?” Philosophical Review 83 (4): 435–450. Google Scholar logo with link to Google Scholar
Nolan, Christopher, dir. 2014. Interstellar. Hollywood, Calif.: Paramount Pictures.Google Scholar logo with link to Google Scholar
Ouyang, Long, Jeff Wu, Xu Jiang, et al. 2022. “Training Language Models to Follow Instructions with Human Feedback.” Advances in Neural Information Processing Systems 351: 27730–27744. Red Hook, NY: Curran Associates. Google Scholar logo with link to Google Scholar
Patterson, David, Joseph Gonzalez, Quoc Le, et al. 2021. “Carbon Emissions and Large Neural Network Training.” Google Scholar logo with link to Google Scholar
Picard, Rosalind W. 2000. Affective Computing. Cambridge, Mass.: MIT Press. Google Scholar logo with link to Google Scholar
Radford, Alec, Jong Wook Kim, Chris Hallacy, et al. 2021. “Learning Transferable Visual Models from Natural Language Supervision.” In Proceedings of the 38th International Conference on Machine Learning, vol. 1391, 8748–8763. Proceedings of Machine Learning Research.Google Scholar logo with link to Google Scholar
Reeves, Byron, and Clifford Nass. 1996. The Media Equation: How People Treat Computers, Television, and New Media Like Real People and Places. Cambridge: Cambridge University Press.Google Scholar logo with link to Google Scholar
Reinhardt, Karoline. 2023. “Trust and Trustworthiness in AI Ethics.” AI and Ethics 3 (3): 735–744. Google Scholar logo with link to Google Scholar
Rettler, Bradley, and Andrew M. Bailey. 2023. “Object.” In The Stanford Encyclopedia of Philosophy (Winter 2023 ed.), ed. by Edward N. Zalta, and Uri Nodelman. [URL]
Rus, Daniela, and Michael T. Tolley. 2015. “Design, Fabrication and Control of Soft Robots.” Nature 521 (7553): 467–475. Google Scholar logo with link to Google Scholar
Scassellati, Brian. 2002. “Theory of Mind for a Humanoid Robot.” Autonomous Robots 12 (1): 13–24. Google Scholar logo with link to Google Scholar
Schegloff, Emanuel A., Gail Jefferson, and Harvey Sacks. 1977. “The Preference for Self-Correction in the Organization of Repair in Conversation.” Language 53 (2): 361–382. Google Scholar logo with link to Google Scholar
Shapira, Natalie, Mosh Levy, Seyed Hossein Alavi, et al. 2024. “Clever Hans or Neural Theory of Mind? Stress Testing Social Reasoning in Large Language Models.” In Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers), 2257–2273. St. Julian’s, Malta: Association for Computational Linguistics. Google Scholar logo with link to Google Scholar
Shoemaker, Sydney. 1975. “Functionalism and Qualia.” Philosophical Studies 27 (5): 291–315. Google Scholar logo with link to Google Scholar
Sider, Theodore. 2013. “Against Parthood.” In Oxford Studies in Metaphysics, vol. 81, ed. by Karen Bennett, and Dean W. Zimmerman, 237–293. Oxford: Oxford University Press.Google Scholar logo with link to Google Scholar
Sravanthi, Settaluri Lakshmi, Meet Doshi, Pavan Tankala, Rudra Murthy, Raj Dabre, and Pushpak Bhattacharyya. 2024. “PUB: A Pragmatics Understanding Benchmark for Assessing LLMs’ Pragmatics Capabilities.” In Findings of the Association for Computational Linguistics: ACL 2024, 12075–12097. Bangkok, Thailand: Association for Computational Linguistics. Google Scholar logo with link to Google Scholar
Stolcke, Andreas, Klaus Ries, Noah Coccaro, et al. 2000. “Dialogue Act Modeling for Automatic Tagging and Recognition of Conversational Speech.” Computational Linguistics 26 (3): 339–373. Google Scholar logo with link to Google Scholar
Strachan, James W. A., Dalila Albergo, Giulia Borghini, et al. 2024. “Testing Theory of Mind in Large Language Models and Humans.” Nature Human Behaviour 8 (7): 1285–1295. Google Scholar logo with link to Google Scholar
Strubell, Emma, Ananya Ganesh, and Andrew McCallum. 2019. “Energy and Policy Considerations for Deep Learning in NLP.” In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, 3645–3650. Florence, Italy: Association for Computational Linguistics. Google Scholar logo with link to Google Scholar
Tian, Katherine, Eric Mitchell, Allan Zhou, et al. 2023. “Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedback.” In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, 5433–5442. Singapore: Association for Computational Linguistics. Google Scholar logo with link to Google Scholar
Tomasello, Michael. 2008. Origins of Human Communication. Cambridge, Mass.: MIT Press. Google Scholar logo with link to Google Scholar
Traum, David R. 1994. “A Computational Theory of Grounding in Natural Language Conversation.” PhD diss., University of Rochester.
Tye, Michael. 2021. “Qualia.” In The Stanford Encyclopedia of Philosophy (Fall 2021 ed.), ed. by Edward N. Zalta. [URL]
Ullman, Tomer D. 2023. “Large Language Models Fail on Trivial Alterations to Theory-of-Mind Tasks.” Google Scholar logo with link to Google Scholar
von Eschenbach, Warren J. 2021. “Transparency and the Black Box Problem: Why We Do Not Trust AI.” Philosophy & Technology 34 (4): 1607–1622. Google Scholar logo with link to Google Scholar
Wittgenstein, Ludwig. (1953) 1958. Philosophical Investigations. Translated by G. E. M. Anscombe. 2nd ed. Oxford: Blackwell.Google Scholar logo with link to Google Scholar
Zemanek, Heinz. 1966. “Semiotics and Programming Languages.” Communications of the ACM 9 (3): 139–143. Google Scholar logo with link to Google Scholar
Zitkovich, Brianna, Tianhe Yu, Sichun Xu, et al. 2023. “RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control.” In Proceedings of the 7th Conference on Robot Learning, vol. 2291, 2165–2183. Proceedings of Machine Learning Research.Google Scholar logo with link to Google Scholar
Mobile Menu Logo with link to supplementary files background Layer 1 prag Twitter_Logo_Blue