Cover not available
Article published In:
Interaction Studies
Vol. 23:1 (2022) ► pp.116–142

Interacting with an embodied interface

Effects of embodied agent and voice command on smart TV interface

| Nanyang Technological University
| Hallym University
| Sogang University
Despite their potential for facilitating interaction between a user and computer, an embodied agent and voice command have not been examined enough for their matching effects. The current study proposes that an embodied agent and voice command generate positive evaluative outcomes, particularly when they are accompanied by each other. To test this prediction, we conducted a 2 (visual output: embodied agent vs. geometric figure) × 2 (input modality: voice command vs. remote controller) between-subjects experiment (N = 52), and examined whether visual output and input modality jointly influence participants’ social attribution (i.e., anthropomorphism, animacy, likability, and perceived intelligence), social presence, and satisfaction. Results show that voice command does facilitate users’ social attribution and social presence, but only when an embodied agent was presented. Also, the effects of voice command on social presence and satisfaction were mediated by anthropomorphism and perceived intelligence respectively, but only when the interface displayed an embodied agent. The present study evidences the holistic nature of human-computer interaction, revealing the importance of matches in the input and output interface.
Keywords: smart TV, embodied agent, voice control, social attribution, social presence, satisfaction
Article outline
  • 1.Introduction
  • 2.Literature review
    • 2.1Effects of embodied agents on a computer interface
    • 2.2Input modality of a smart TV
    • 2.3Contingent effect of visual output and input modality
  • 3.Method
    • 3.1Design and participants
    • 3.2Procedure
    • 3.3Material
    • 3.4Manipulation
    • 3.5Measures
      • Attribution of social characters
      • Social presence
      • Satisfaction
  • 4.Results
    • 4.1Effects on social attribution
    • 4.2Effects on social presence and satisfaction
    • 4.3Conditional indirect effect of input modality
  • 5.Discussion
    • 5.1Summary of findings
    • 5.2Implications
    • 5.3Limitations and directions for further research
  • Disclosure statement
  • References
Published online: 20 October 2022
DOI logo
https://doi.org/10.1075/is.20030.lee
References (47)
References
Allison, F., Luger, E., & Hofmann, K. (2017). Spontaneous Interactions with a Virtually Embodied Intelligent Assistant in Minecraft. Proceedings of the 2017 CHI Conference Extended Abstracts on Human Factors in Computing Systems, 2337–2344. DOI logoGoogle Scholar
Bartneck, C., Kulić, D., Croft, E., & Zoghbi, S. (2009). Measurement instruments for the anthropomorphism, animacy, likeability, perceived intelligence, and perceived safety of robots. International Journal of Social Robotics, 1(1), 71–81. DOI logoGoogle Scholar
Baylor, A. L., & Ryu, J. (2003). The effects of image and animation in enhancing pedagogical agent persona. Journal of Educational Computing Research, 28 1, 373–394. DOI logoGoogle Scholar
Brave, S., Nass, C., & Hutchinson, K. (2005). Computers that care: Investigating the effects of orientation of emotion exhibited by an embodied computer agent. International Journal of Human-Computer Studies, 62 1, 161–178. DOI logoGoogle Scholar
Brennan, S. E., & Hulteen, E. A. (1995). Interaction and feedback in a spoken language system: A theoretical framework. Knowledge-Based Systems, 8 1, 143–151. DOI logoGoogle Scholar
Clark, H. H. (1999). How do real people communicate with virtual partners. Proceedings of 1999 AAAI Fall Symposium, Psychological Models of Communication in Collaborative Systems, 43–47. North Falmouth, MA.Google Scholar
Cohen, P. R., & Oviatt, S. L. (1995). The role of voice input for human-machine communication. Proceedings of the National Academy of Sciences, 92 1, 9921–9927. Retrieved from [URL]. DOI logo
Cordasco, G., Esposito, M., Masucci, F., Riviello, M. T., Esposito, A., Chollet, G., … Pelosi, G. (2014). Assessing voice user interfaces: The vAssist system prototype. 5th IEEE Conference on Cognitive Infocommunications, 91–96. DOI logo
Dehn, D. M., & Van Mulken, S. (2000). The impact of animated interface agents: A review of empirical research. International Journal of Human-Computer Studies, 52 1, 1–22. DOI logoGoogle Scholar
Dworkin, M., Chakraborty, A., Lee, S., Monahan, C., Hightow-Weidman, L., Garofalo, R., … Jimenez, A. (2018). A realistic talking human embodied agent mobile phone intervention to promote hiv medication adherence and retention in care in young HIV-positive african american men who have sex with men: Qualitative study. JMIR MHealth and UHealth, 6 1, e10211. DOI logoGoogle Scholar
Fogg, B., & Nass, C. (1997). How users reciprocate to computers: An experiment that demonstrates behavior change. Proceedings of the CHI ’97 Extended Abstracts on Human Factors in Computing Systems, 331–332. DOI logoGoogle Scholar
Hasegawa, D., Cassell, J., & Araki, K. (2010). The role of embodiment and perspective in direction-giving systems. 2010 AAAI Fall Symposium. Retrieved from [URL]
Hayes, A. F. (2013). Introduction to mediation, moderation, and conditional process analysis: A regression-based approach. New York, NY, US: Guilford Press.Google Scholar
Hong, W., Hess, T. J., & Hardin, A. (2013). When filling the wait makes it feel longer: A paradigm shift perspective for managing online delay. MIS Quarterly, 37 1, 383–406. Retrieved from [URL]. DOI logo
Hutchins, E. L., Hollan, J. D., & Norman, D. A. (1985). Direct manipulation interfaces. Human–Computer Interaction, 1 1, 311–338. DOI logoGoogle Scholar
Jeong, J.-W., & Lee, D.-H. (2014). Inferring search intents from remote control movement patterns: A new content search method for smart TV. IEEE Transactions on Consumer Electronics, 60 1, 92–98. DOI logoGoogle Scholar
Kätsyri, J., Förger, K., Mäkäräinen, M., & Takala, T. (2015). A review of empirical evidence on different uncanny valley hypotheses: Support for perceptual mismatch as one road to the valley of eeriness. Frontiers in Psychology, 6 1, 390. DOI logoGoogle Scholar
Kim, Yanghee, Thayne, J., & Wei, Q. (2017). An embodied agent helps anxious students in mathematics learning. Educational Technology Research and Development, 65 1, 219–235. DOI logoGoogle Scholar
Kim, Youjeong, & Sundar, S. S. (2012). Anthropomorphism of computers: Is it mindful or mindless? Computers in Human Behavior, 28 1, 241–250. DOI logoGoogle Scholar
Koda, T., & Maes, P. (1996). Agents with faces: The effect of personification. Proceedings of 5th International Workshop on Robot and Human Communication, 189–194. DOI logoGoogle Scholar
Lee, B., Isenberg, P., Riche, N. H., & Carpendale, S. (2012). Beyond Mouse and Keyboard: Expanding Design Considerations for Information Visualization Interactions. IEEE Transactions on Visualization and Computer Graphics, 18 1, 2689–2698. DOI logoGoogle Scholar
Lee, D., Sah, Y. J., & Lee, S. (2019). Improving usability perception of error-prone ai speakers: Elaborated feedback mitigates negative consequences of errors. International Journal of Human–Computer Interaction, 35 1, 1645–1652. DOI logoGoogle Scholar
Lee, K. M. (2004). Presence, explicated. Communication Theory, 14 (1), 27–50. DOI logoGoogle Scholar
Lee, K. M., & Nass, C. (2005). Social-psychological origins of feelings of presence: Creating social presence with machine-generated voices. Media Psychology, 7 1, 31–45. DOI logoGoogle Scholar
Lee, K. M., Peng, W., Jin, S.-A., & Yan, C. (2006). Can robots manifest personality?: An empirical test of personality recognition, social responses, and social presence in human–robot interaction. Journal of Communication, 56 1, 754–772. DOI logoGoogle Scholar
Mitchell, W. J., Szerszen, K. A., Lu, A. S., Schermerhorn, P. W., Scheutz, M., & MacDorman, K. F. (2011). A mismatch in the human realism of face and voice produces an uncanny valley. I-Perception, 2 1, 10–12. DOI logoGoogle Scholar
Mori, M. (1970). The uncanny valley. Energy, 7 (4), 33–35.Google Scholar
Nass, C., & Gong, L. (2000). Speech interfaces from an evolutionary perspective. Communications of the ACM, 43 (9), 36–43. DOI logoGoogle Scholar
Nass, C., & Moon, Y. (2000). Machines and mindlessness: Social responses to computers. Journal of Social Issues, 56 1, 81–103. DOI logoGoogle Scholar
Nass, C., Moon, Y., & Carney, P. (1999). Are people polite to computers? Responses to computer-based interviewing systems. Journal of Applied Social Psychology, 29 1, 1093–1109. DOI logoGoogle Scholar
Nass, C., Moon, Y., & Green, N. (1997). Are machines gender neutral? Gender-stereotypic responses to computers with voices. Journal of Applied Social Psychology, 27 1, 864–876. DOI logoGoogle Scholar
Nass, C., Steuer, J., & Tauber, E. R. (1994). Computers are social actors. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, 72–78. DOI logoGoogle Scholar
Pyae, A., & Joelsson, T. N. (2018). Investigating the usability and user experiences of voice user interface: A case of google home smart speaker. Proceedings of the 20th International Conference on Human-Computer Interaction with Mobile Devices and Services Adjunct, 127–131. DOI logoGoogle Scholar
Rhee, E., Shin, I., Jung, Y., & Lee, H. (2013). Cloud-based 3D gaming system for smart TV. Proceedings of the 2013 International Conference on ICT Convergence (ICTC), 714–715. DOI logoGoogle Scholar
Rosenberg-Kima, R. B., Baylor, A. L., Plant, E. A., & Doerr, C. E. (2007). The importance of interface agent visual presence: Voice alone is less effective in impacting young women’s attitudes toward engineering. In Y. de Kort, W. IJsselsteijn, C. Midden, B. Eggen, & B. J. Fogg (Eds.), Persuasive Technology (pp. 214–222). Berlin: Springer. DOI logoGoogle Scholar
Sah, Y. J., & Peng, W. (2015). Effects of visual and linguistic anthropomorphic cues on social perception, self-awareness, and information disclosure in a health website. Computers in Human Behavior, 45 1, 392–401. DOI logoGoogle Scholar
Seyama, J., & Nagayama, R. S. (2007). The uncanny valley: Effect of realism on the impression of artificial human faces. Presence: Teleoperators and Virtual Environments, 16 1, 337–351. DOI logoGoogle Scholar
Shamekhi, A., Liao, Q. V., Wang, D., Bellamy, R. K. E., & Erickson, T. (2018). Face value? Exploring the effects of embodiment for a group facilitation agent. Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems, 1–13. DOI logoGoogle Scholar
Silvia, P. J., & Duval, T. S. (2001). Objective self-awareness theory: Recent progress and enduring problems. Personality and Social Psychology Review, 5 1, 230–241. DOI logoGoogle Scholar
Sproull, L., Subramani, M., Kiesler, S., Walker, J. H., & Waters, K. (1996). When the interface is a face. Human–Computer Interaction, 11 1, 97–124. DOI logoGoogle Scholar
Stafford, R. Q., MacDonald, B. A., Jayawardena, C., Wegner, D. M., & Broadbent, E. (2014). Does the robot have a mind? Mind perception and attitudes towards robots predict use of an eldercare robot. International Journal of Social Robotics, 6 1, 17–32. DOI logoGoogle Scholar
Walther, J. B. (1996). Computer-mediated communication: Impersonal, interpersonal, and hyperpersonal interaction. Communication Research, 23 1, 3–43. DOI logoGoogle Scholar
(2011). Visual cues in computer-mediated communication: Sometimes less is more. In A. Kappas & N. C. Krämer (Eds.), Face-to-Face Communication over the Internet (pp. 17–38). Cambridge, UK: Cambridge University Press. DOI logoGoogle Scholar
Walther, J. B., Slovacek, C. L., & Tidwell, L. C. (2001). Is a picture worth a thousand words? Photographic images in long-term and short-term computer-mediated communication. Communication Research, 28 (1), 105–134. DOI logoGoogle Scholar
Yee, N., Bailenson, J. N., & Rickertsen, K. (2007). A meta-analysis of the impact of the inclusion and realism of human-like faces on user experiences in interfaces. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, 1–10. DOI logoGoogle Scholar
Yi, J. S., Kang, Y. a, & Stasko, J. (2007). Toward a Deeper Understanding of the Role of Interaction in Information Visualization. IEEE Transactions on Visualization and Computer Graphics, 13 1, 1224–1231. DOI logoGoogle Scholar
Yu, E., Hong, A., & Hwang, J. (2016). A socio-technical analysis of factors affecting the adoption of smart TV in Korea. Computers in Human Behavior, 61 1, 89–102. DOI logoGoogle Scholar