Cover not available
Article published In:
Interaction Studies
Vol. 18:2 (2017) ► pp.254–275

An experimental approach to study the physiology of natural social interactions

| Institut de Neurosciences de la Timone |

The classical experimental methodology is ill-suited for the investigation of the behavioral and physiological correlates of natural social interactions. A new experimental approach combining a natural conversation between two persons with control conditions is proposed in this paper. Behavior, including gaze direction and speech, and physiology, including electrodermal activity, are recorded during a discussion between two participants through videoconferencing. Control for the social aspect of the interaction is provided by the use of an artificial agent and of videoed conditions. A cover story provides spurious explanations for the purpose of the experiment and for the recordings, as well as a controlled and engaging topic of discussion. Preprocessing entails transforming raw measurements into boxcar and delta functions time series indicating when a certain behaviour or physiological event is present. The preliminary analysis presented here consists in finding statistically significant differences between experimental conditions in the temporal associations between behavioral and physiological time series. Significant results validate the experimental approach and further developments including more elaborate analysis and adaptation of the paradigm to functional MRI are discussed.

Keywords: social interactions, conversation, physiology, artificial agent, embodied conversational agent
Article outline
  • 1.Introduction
    • 2.Summary of the experimental approach
  • 3.Methods
    • 3.1Participants
    • 3.2Experimental paradigm
      • 3.2.1Cover story
      • 3.2.2Experimental conditions
    • 3.3Experimental setup
      • 3.3.1Embodied conversational agent
      • 3.3.2Physical setup
    • 3.4Experimental recording
    • 3.5Data preprocessing
      • 3.5.1Electrodermal activity
      • 3.5.2Eyetracking
      • 3.5.3Conversation behaviour
    • 3.6Statistical analysis
  • 4.Results
    • 4.1Effect of temporal resolution of the analysis
    • 4.2Relation with verbal behaviour
    • 4.3Relation with oculomotor behaviours
  • 5.Discussion
    • 5.1Current findings
    • 5.2Further developments
  • Acknowledgements
  • References
Published online: 8 December 2017
DOI logo
https://doi.org/10.1075/is.18.2.06gry
References
Bach, D. R., & Friston, K. J.
(2013) Model-based analysis of skin conductance responses: Towards causal models in psychophysiology. Psychophysiology, 50(1), 15–22. DOI logoGoogle Scholar
Benedek, M., & Kaernbach, C.
(2010) Decomposition of skin conductance data by means of nonnegative deconvolution. Psychophysiology, 47(4), 647–658.Google Scholar
Bigi, B., Watanabe, T., & Prévot, L.
(2014) Representing multimodal linguistics annotated data. Proceedings of 9th International conference on Language Resources and Evaluation (LREC) , Reykjavik (Iceland).
Chaminade, T., Hodgins, J., & Kawato, M.
(2007) Anthropomorphism influences perception of computer-animated characters' actions. Social Cognitive and Affective Neuroscience, 2(3), 206–216. DOI logoGoogle Scholar
Chaminade, T., & Cheng, G.
(2009) Social cognitive neuroscience and humanoid robotics. Journal of Physiology-Paris, 103(3-5), 286–295. DOI logoGoogle Scholar
Chaminade, T., Rosset, D., Fonseca, D. D., Nazarian, B., Lutcher, E., Cheng, G., & Deruelle, C.
(2012) How do we think machines think? An fMRI study of alleged competition with an artificial intelligence. Frontiers in Human Neuroscience, 6, 103.Google Scholar
Chaminade, T., Fonseca, D., Rosset, D., Cheng, G., & Deruelle, C.
(2015) Atypical modulation of hypothalamic activity by social context in ASD. Research in Autism Spectrum Disorders, 101, 41–50. DOI logoGoogle Scholar
Chartrand, T. L., & Bargh, J. A.
(1999) The chameleon effect: The perception–behavior link and social interaction. Journal of personality and social psychology, 76(6), 893–910. DOI logoGoogle Scholar
Dawson, M. E., Schell, A. M., & Filion, D. L.
(2007) The electrodermal system. In J. T. Cacioppo, L. G. Tassinary, & G. G. Berntson (Eds.). Handbook of psychophysiology (3rd ed.; pp. 159–181). Cambridge, UK: Cambridge University Press. DOI logoGoogle Scholar
Dennett, D. C.
(1996) The Intentional Stance (6th printing). Cambridge, MA, USA: The MIT Press.Google Scholar
Diehl, J. J., Schmitt, L. M., Villano, M., & Crowell, C. R.
(2012) The clinical use of robots for individuals with Autism Spectrum Disorders: A critical review. Research in Autism Spectrum Disorders, 6(1), 249–262. DOI logoGoogle Scholar
French, D. P., & Sutton, S.
(2010) Reactivity of measurement in health psychology: How much of a problem is it? What can be done about it? British Journal of Health Psychology, 15(3), 453–468. DOI logoGoogle Scholar
Frith, C. D., & Allen, H. A.
(1983) The skin conductance orienting response as an index of attention. Biological psychology, 17(1), 27–39. DOI logoGoogle Scholar
Hauber, J., Regenbrecht, H., Hills, A., Cockburn, A., & Billinghurst, M.
(2005) Social presence in two-and three-dimensional videoconferencing. Proceedings of 8th Annual International Workshop on Presence , London (UK), 189–198.
Khalfa, S., Isabelle, P., Jean-Pierre, B., & Manon, R.
(2002) Event-related skin conductance responses to musical emotions in humans. Neuroscience letters, 328(2), 145–149. DOI logoGoogle Scholar
Kilpatrick, D. G.
(1972) Differential responsiveness of two electrodermal indices to psychological stress and performance of a complex cognitive task. Psychophysiology, 9(2), 218–226. DOI logoGoogle Scholar
Krach, S., Hegel, F., Wrede, B., Sagerer, G., Binkofski, F., & Kircher, T.
(2008) Can Machines Think? Interaction and Perspective Taking with Robots Investigated via fMRI. PLoS ONE, 3(7), e2597. DOI logoGoogle Scholar
Kuster, C., Popa, T., Bazin, J.-C., Gotsman, C., & Gross, M.
(2012) Gaze correction for home video conferencing. ACM Transactions on Graphics, 31(6), 174. DOI logoGoogle Scholar
Laming, D. R. J.
(1968) Information theory of choice-reaction times. London: Academic Press.Google Scholar
Nomura, T., Suzuki, T., Kanda, T., & Kato, K.
(2006) Measurement of negative attitudes toward robots. Interaction Studies, 7(3), 437–454 DOI logoGoogle Scholar
Ochs, M., Niewiadomski, R., Brunet, P., & Pelachaud, C.
(2012) Smiling virtual agent in social context. Cognitive Processing, 13(S2), 519–532. DOI logoGoogle Scholar
Pelachaud, C.
(2009) Modelling multimodal expression of emotion in a virtual agent. Philosophical Transactions of the Royal Society B: Biological Sciences, 364(1535), 3539–3548. DOI logoGoogle Scholar
Pfungst, O.
(1911) Clever Hans (The horse of Mr. von Osten): A contribution to experimental animal and human psychology (Trans. C. L. Rahn). New York: Henry Holt. DOI logoGoogle Scholar
Riek, L. D.
(2012) Wizard of oz studies in hri: A systematic review and new reporting guidelines. Journal of Human-Robot Interaction 11.Google Scholar
Rolison, M. J., Naples, A. J., & McPartland, J. C.
(2015) Interactive Social Neuroscience to Study Autism Spectrum Disorder. The Yale Journal of Biology and Medicine, 88(1), 17–24.Google Scholar
Roth, W. T., Dawson, M. E., & Filion, D. L.
(2012) Publication recommendations for electrodermal measurements. Psychophysiology, 491, 1017–1034. DOI logoGoogle Scholar
Schilbach, L.
(2010) A second-person approach to other minds. Nature Reviews Neuroscience, 11(6), 449–449. DOI logoGoogle Scholar
Schilbach, L., Timmermans, B., Reddy, V., Costall, A., Bente, G., Schlicht, T., & Vogeley, K.
(2013) Toward a second-person neuroscience. Behavioral And Brain Sciences, 36(4), 393–414. DOI logoGoogle Scholar
Senju, A., & Johnson, M. H.
(2009) The eye contact effect: Mechanisms and development. Trends in Cognitive Sciences, 13(3), 127–134. DOI logoGoogle Scholar
Turing, A.
(1950), Computing machinery and intelligence. Mind, 59(236), 433–460. DOI logoGoogle Scholar
Wykowska, A., Chaminade, T., & Cheng, G.
(2016) Embodied artificial agents for understanding human social cognition. Philosophical Transactions of the Royal Society Biological Sciences, 371(1693), 20150375. DOI logoGoogle Scholar
Xiong, X., & De la Torre, F.
(2013) Supervised descent method and its application to face alignment. IEEE Conference on Computer Vision and Pattern Recognition (CVPR).Google Scholar
Cited by

Cited by 7 other publications

Chaminade, Thierry, Birgit Rauchbauer, Bruno Nazarian, Morgane Bourhis, Magalie Ochs & Laurent Prévot
2018. Proceedings of the International Conference on Multimodal Interaction Adjunct - ICMI '18, ► pp. 1 ff. DOI logo
Chaminade, Thierry, Birgit Rauchbauer, Bruno Nazarian, Morgane Bourhis, Magalie Ochs & Laurent Prévot
2018. Proceedings of the 6th International Conference on Human-Agent Interaction, ► pp. 333 ff. DOI logo
Rauchbauer, Birgit, Bruno Nazarian, Morgane Bourhis, Magalie Ochs, Laurent Prévot & Thierry Chaminade
2019. Brain activity during reciprocal social interaction investigated using conversational robots as control condition. Philosophical Transactions of the Royal Society B: Biological Sciences 374:1771 ► pp. 20180033 ff. DOI logo
Riou, Matthieu, Bassam Jabaian, Stephane Huet & Fabrice Lefevre
2019. ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), ► pp. 3072 ff. DOI logo
Riou, Matthieu, Bassam Jabaian, Stéphane Huet & Fabrice Lefèvre
2019. Lilia, A Showcase for Fast Bootstrap of Conversation-Like Dialogues Based on a Goal-Oriented System. In Statistical Language and Speech Processing [Lecture Notes in Computer Science, 11816], ► pp. 31 ff. DOI logo
Rudnicki, Konrad, Carolyn Declerck, Charlotte De Backer & Mario Berth
2019. Physiological changes during first encounters and their role in determining the perceived interaction quality. Interaction Studies. Social Behaviour and Communication in Biological and Artificial Systems 20:2 ► pp. 275 ff. DOI logo
Nicolas Spatola
2019. L’interaction Homme-Robot, de l’anthropomorphisme à l’humanisation. L’Année psychologique Vol. 119:4 ► pp. 515 ff. DOI logo

This list is based on CrossRef data as of 15 april 2022. Please note that it may not be complete. Sources presented here have been supplied by the respective publishers. Any errors therein should be reported to them.