Papers
Topics
Authors
Recent
Search
2000 character limit reached

Engaged and Affective Virtual Agents: Their Impact on Social Presence, Trustworthiness, and Decision-Making in the Group Discussion

Published 20 Aug 2023 in cs.HC | (2308.10385v3)

Abstract: This study investigates how different virtual agent (VA) behaviors influence subjects' perceptions and group decision-making. Participants carried out experimental group discussions with a VA exhibiting varying levels of engagement and affective behavior. Engagement refers to the VA's focus on the group task, whereas affective behavior reflects the VA's emotional state. The findings revealed that VA's engagements effectively captured participants' attention even in the group setting and enhanced group synergy, thereby facilitating more in-depth discussion and producing better consensus. On the other hand, VA's affective behavior negatively affected the perceived social presence and trustworthiness. Consequently, in the context of group discussion, participants preferred the engaged and non-affective VA to the non-engaged and affective VA. The study provides valuable insights for improving the VA's behavioral design as a team member for collaborative tasks.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (119)
  1. Ashimi Rashidat Abiodun. 2014. Organizational conflicts: Causes, effects and remedies. International Journal of Academic Research in Economics and Management Sciences 3, 6 (2014), 118.
  2. Touching Virtual Humans: Haptic Responses Reveal the Emotional Impact of Affective Agents. IEEE Transactions on Affective Computing 14, 1 (2023), 331–342. https://doi.org/10.1109/TAFFC.2020.3038137
  3. Ghazanfar Ali and Jae-In Hwang. 2022. Improving Co-Speech Gesture Rule-Map Generation via Wild Pose Matching with Gesture Units.. In SIGGRAPH Asia 2022 Posters (SA ’22). ACM, Article 4, 2 pages. https://doi.org/10.1145/3550082.3564185
  4. Automatic text-to-gesture rule generation for embodied conversational agents. Computer Animation and Virtual Worlds 31, 4-5 (2020), e1944. https://doi.org/10.1002/cav.1944
  5. Jeremy N Bailenson and Andrew C Beall. 2006. Transformed social interaction: Exploring the digital plasticity of avatars. In Avatars at work and play: Collaboration and interaction in shared virtual environments. Springer, 1–16.
  6. Interpersonal distance in immersive virtual environments. Personality and social psychology bulletin 29, 7 (2003), 819–833.
  7. Andrew Scott Baron and Yarrow Dunham. 2015. Representing ‘us’ and ‘them’: Building blocks of intergroup cognition. Journal of Cognition and Development 16, 5 (2015), 780–801.
  8. Frank W Bond and David Bunce. 2003. The role of acceptance and job control in mental health, job satisfaction, and work performance. Journal of applied psychology 88, 6 (2003), 1057.
  9. Sharon S Brehm and Jack W Brehm. 2013. Psychological reactance: A theory of freedom and control. Academic Press.
  10. The effect of group size on people’s attitudes and cooperative behaviors toward robots in interactive gameplay. In 2012 IEEE RO-MAN: The 21st IEEE International Symposium on Robot and Human Interactive Communication. 845–850. https://doi.org/10.1109/ROMAN.2012.6343857
  11. Affective engagement to emotional facial expressions of embodied social agents in a decision-making game. Computer Animation and Virtual Worlds (2012).
  12. Exploring the Social Influence of Virtual Humans Unintentionally Conveying Conflicting Emotions. In 2023 IEEE Conference Virtual Reality and 3D User Interfaces (VR). 571–580. https://doi.org/10.1109/VR55154.2023.00072
  13. Nicholas Clarke. 2010. Emotional intelligence abilities and their relationships with team processes. Team Performance Management: An International Journal 16, 1/2 (2010), 6–32.
  14. William Jay Conover and Ronald L Iman. 1979. On multiple-comparisons procedures. Los Alamos Sci. Lab. Tech. Rep. LA-7677-MS 1 (1979), 14.
  15. Mary-Beth Cooper and Heath Boice-Pardee. 2011. Managing conflict from the middle. New directions for student services 2011, 136 (2011), 35–42.
  16. Social facilitation of dominant responses by the presence of an audience and the mere presence of others. Journal of personality and social psychology 9, 3 (1968), 245.
  17. Crazy Minnow Studio. 2014. SALSA LipSync. https://crazyminnowstudio.com/ Accessed oct. 14, 2022.
  18. Embodied conversational agent for emotional recognition training. In Proceedings of ThinkMind, ACHI 2020: The Thirteenth International Conference on Advances in Computer-Human Interactions, 21-25 November 2020, Valencia, Spain. 21-25 November 2020.
  19. The impact of emotion displays in embodied agents on emergence of cooperation with people. Presence: teleoperators and virtual environments 20, 5 (2011), 449–465.
  20. Humans versus Computers: Impact of Emotion Expressions on People’s Decision Making. IEEE Transactions on Affective Computing 6, 2 (2015), 127–136. https://doi.org/10.1109/TAFFC.2014.2332471
  21. SimSensei Kiosk: A virtual human interviewer for healthcare decision support. In Proceedings of the 2014 international conference on Autonomous agents and multi-agent systems. 1061–1068.
  22. You talkin’to me? A practical attention-aware embodied agent. In Human-Computer Interaction–INTERACT 2019: 17th IFIP TC 13 International Conference, Paphos, Cyprus, September 2–6, 2019, Proceedings, Part III 17. Springer, 760–780.
  23. An Aligned Rank Transform Procedure for Multifactor Contrast Tests. In The 34th Annual ACM Symposium on User Interface Software and Technology (Virtual Event, USA) (UIST ’21). Association for Computing Machinery, New York, NY, USA, 754–768. https://doi.org/10.1145/3472749.3474784
  24. Aaron C Elkins and Douglas C Derrick. 2013. The sound of trust: voice as a measurement of trust during interactions with embodied conversational agents. Group decision and negotiation 22, 5 (2013), 897–913.
  25. Statistical power analyses using G* Power 3.1: Tests for correlation and regression analyses. Behavior research methods 41, 4 (2009), 1149–1160.
  26. Social functions of emotion and emotion regulation. Handbook of emotions 4 (2016), 424–439.
  27. Avatars versus agents: a meta-analysis quantifying the effect of agency on social influence. Human–Computer Interaction 30, 5 (2015), 401–432.
  28. Milton Friedman. 1940. A comparison of alternative tests of significance for the problem of m rankings. The annals of mathematical statistics 11, 1 (1940), 86–92.
  29. MET-iquette: enabling virtual agents to have a social compliant behavior in the Metaverse. In 2022 16th International Conference on Signal-Image Technology & Internet-Based Systems (SITIS). IEEE, 394–401.
  30. Alicia A Grandey. 2000. Emotional regulation in the workplace: A new way to conceptualize emotional labor. Journal of occupational health psychology 5, 1 (2000), 95.
  31. Can virtual humans be more engaging than real ones?. In Human-Computer Interaction. HCI Intelligent Multimodal Interaction Environments: 12th International Conference, HCI International 2007, Beijing, China, July 22-27, 2007, Proceedings, Part III 12. Springer, 286–297.
  32. James J Gross and Robert W Levenson. 1993. Emotional suppression: physiology, self-report, and expressive behavior. Journal of personality and social psychology 64, 6 (1993), 970.
  33. ET Hall. 1966. The Hidden Dimension: Man’s Use of Space in Public and Private (London, The Bodley Head). (1966).
  34. Jay Hall and Wilfred Harvey Watson. 1970. The effects of a normative intervention on group decision-making performance. Human relations 23, 4 (1970), 299–317.
  35. Stephen G Harkins. 1987. Social loafing and social facilitation. Journal of experimental social psychology 23, 1 (1987), 1–18.
  36. Chad Harms and Frank Biocca. 2006. Internal Consistency and Reliability of the Networked Minds Social Presence Measure.
  37. Sandra G Hart. 2006. NASA-task load index (NASA-TLX); 20 years later. In Proceedings of the human factors and ergonomics society annual meeting, Vol. 50. Sage publications Sage CA: Los Angeles, CA, 904–908.
  38. Group decision support: The effects of designated human leaders and statistical feedback in computerized conferences. Journal of Management Information Systems 8, 2 (1991), 81–108.
  39. Timothy R Hinkin and Chester A Schriesheim. 1994. An examination of subordinate-perceived relationships between leader reward and punishment behavior and leader bases of power. Human Relations 47, 7 (1994), 779–800.
  40. Mach: My automated conversation coach. In Proceedings of the 2013 ACM international joint conference on Pervasive and ubiquitous computing. 697–706.
  41. I just wanna blame somebody, not something! Reactions to a computer agent giving negative feedback based on the instructions of a person. International Journal of Human-Computer Studies 154 (2021), 102683.
  42. Virtual Rapport 2.0. In International workshop on intelligent virtual agents.
  43. Foundations for an empirically determined scale of trust in automated systems. International journal of cognitive ergonomics 4, 1 (2000), 53–71.
  44. David W Johnson and Frank P Johnson. 1991. Joining together: Group theory and group skills. Prentice-Hall, Inc.
  45. Expressing social attitudes in virtual agents for social coaching. In Workshop Affects, Compagnons Artificiels et Interaction.
  46. Vahid Kazemi and Josephine Sullivan. 2014. One millisecond face alignment with an ensemble of regression trees. In Proceedings of the IEEE conference on computer vision and pattern recognition. 1867–1874.
  47. Building interactive virtual humans for training environments. In Proceedings of i/itsec, Vol. 174. 911–916.
  48. The Impacts of Visual Effects on User Perception With a Virtual Human in Augmented Reality Conflict Situations. IEEE Access 9 (2021), 35300–35312. https://doi.org/10.1109/ACCESS.2021.3062037
  49. Does a Digital Assistant Need a Body? The Influence of Visual Embodiment and Social Behavior on the Perception of Intelligent Virtual Agents in AR. In 2018 IEEE International Symposium on Mixed and Augmented Reality. https://doi.org/10.1109/ISMAR.2018.00039
  50. Reducing Task Load with an Embodied Intelligent Virtual Assistant for Improved Performance in Collaborative Decision Making. In 2020 IEEE Conference on Virtual Reality and 3D User Interfaces (VR). 529–538. https://doi.org/10.1109/VR46266.2020.00074
  51. Effects of patient care assistant embodiment and computer mediation on user experience. In 2019 IEEE International Conference on Artificial Intelligence and Virtual Reality. 17–177.
  52. Let’s Face It: Influence of Facial Expressions on Social Presence in Collaborative Virtual Reality. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–16.
  53. Davis E King. 2009. Dlib-ml: A machine learning toolkit. The Journal of Machine Learning Research 10 (2009), 1755–1758.
  54. Emotion feedback in small group collaboration: A research agenda for group emotion management support systems. In Proceedings of the International Conference on Group Decision and Negotiation. 1–12.
  55. Olena Koval and Håvard Hansen. 2021. Together alone: complex tourism decisions in couples and random groups. Anatolia (2021), 1–11.
  56. Michael W Kramer and Jon A Hess. 2002. Communication rules for the display of emotions in organizational settings. Management Communication Quarterly 16, 1 (2002), 66–80.
  57. It’s in their eyes: A study on female and male virtual humans’ gaze. In Intelligent Virtual Agents: 10th International Conference, IVA 2011, Reykjavik, Iceland, September 15-17, 2011. Proceedings 11. Springer, 80–92.
  58. Junyeong Kum and Myungho Lee. 2022. Can Gestural Filler Reduce User-Perceived Latency in Conversation with Digital Humans? Applied Sciences 12, 21 (2022), 10972.
  59. The desert survival problem. Plymouth, Michigan: Experimental Learning Methods.
  60. The wobbly table: Increased social presence via subtle incidental movement of a real-virtual table. In 2016 IEEE Virtual Reality (VR). 11–17. https://doi.org/10.1109/VR.2016.7504683
  61. It’s only a computer: Virtual humans increase willingness to disclose. Computers in Human Behavior 37 (2014), 94–100.
  62. Reporting mental health symptoms: breaking down barriers to care with virtual human interviewers. Frontiers in Robotics and AI 4 (2017), 51.
  63. The extended cohn-kanade dataset (ck+): A complete dataset for action unit and emotion-specified expression. In 2010 ieee computer society conference on computer vision and pattern recognition-workshops.
  64. Visual Attention in Real-World Conversation: Gaze Patterns Are Modulated by Communication and Group Size. Applied Psychology 70, 4 (2021), 1602–1627.
  65. Leader–member exchange (LMX) and performance: A meta-analytic review. Personnel psychology 69, 1 (2016), 67–121.
  66. Four-participant group conversation: A facilitation robot controlling engagement density as the fourth participant. Computer Speech & Language 33, 1 (2015), 1–24.
  67. John W Mauchly. 1940. Significance test for sphericity of a normal n-variate distribution. The Annals of Mathematical Statistics 11, 2 (1940), 204–209.
  68. Collective decision making, leadership, and collective intelligence: Tests with agent-based simulations and a Field study. The Leadership Quarterly 27, 2 (2016), 218–241.
  69. The influence of emotions in embodied agents on human decision-making. In International conference on intelligent virtual agents. Springer, 357–370.
  70. Nicoleta Meslec and Petru Lucian Curşeu. 2013. Too close or too far hurts: cognitive distance and group cognitive synergy. Small Group Research 44, 5 (2013), 471–497.
  71. Mixamo. 2008. Mixamo. https://www.mixamo.com Accessed oct. 14, 2022.
  72. Christos N Moridis and Anastasios A Economides. 2012. Affective learning: Empathetic agents with emotional facial and tone of voice expressions. IEEE Transactions on Affective Computing 3, 3 (2012), 260–272.
  73. Robust eye contact detection in natural multi-person interactions using gaze and speaking behaviour. In Proceedings of the 2018 ACM Symposium on Eye Tracking Research & Applications. 1–10.
  74. An Emotionally Responsive Virtual Parent for Pediatric Nursing Education: A Framework for Multimodal Momentary and Accumulated Interventions. In IEEE International Symposium on Mixed and Augmented Reality.
  75. Naver Cloud. 2019. Naver Cloud Platform AI Services. https://www.ncloud.com/product/aiService/ Accessed oct. 14, 2022.
  76. Vishal Nayak and Matthew Turk. 2005. Emotional expression in virtual agents through body language. In International Symposium on Visual Computing. Springer, 313–320.
  77. Paul M Nemiroff and William A Pasmore. 2001. Lost at sea: A consensus-seeking task. The Pfeiffer book of successful team-building tools: Best of the annuals (2001), 165–172.
  78. Responding to positive emotions at work–the four steps and potential benefits of a validating response to coworkers’ positive experiences. Frontiers in Psychology (2021), 4553.
  79. Empathy in Virtual Agents and Robots: A Survey. ACM Trans. Interact. Intell. Syst. 7, 3, Article 11 (sep 2017), 40 pages. https://doi.org/10.1145/2912150
  80. The avatar will see you now: Support from a virtual human provides socio-emotional benefits. Computers in Human Behavior 136 (2022), 107368.
  81. Hubert Pham. 2006. Pyaudio: Portaudio v19 python bindings. URL: https://people. csail. mit. edu/hubert/pyaudio (2006).
  82. Helping children think: Gaze aversion and teaching. British journal of developmental psychology 24, 3 (2006), 577–588.
  83. How our personality shapes our interactions with virtual characters-implications for research and development. In International Conference on Intelligent Virtual Agents. Springer, 208–221.
  84. Conversations with a virtual human: Synthetic emotions and human responses. Computers in Human Behavior 34 (2014), 58–68.
  85. Eva: Generating emotional behavior of virtual agents using expressive features of gait and gaze. In ACM symposium on applied perception 2019. 1–10.
  86. Matthias Rehm and Elisabeth André. 2005. Where do they look? Gaze behaviors of multiple users interacting with an embodied conversational agent. In International Workshop on Intelligent Virtual Agents. Springer, 241–252.
  87. Nils Reimers and Iryna Gurevych. 2019. Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics. https://arxiv.org/abs/1908.10084
  88. Robot-supported collaborative learning (RSCL): Social robots as teaching assistants for higher education small group facilitation. Frontiers in Robotics and AI 6 (2020), 148.
  89. Laurie A Rudman and Stephanie A Goodwin. 2004. Gender differences in automatic in-group bias: Why do women like women more than men like men? Journal of personality and social psychology 87, 4 (2004), 494.
  90. The effect of virtual agent gender and embodiment on the experiences and performance of students in Virtual Field Trips. In 2020 IEEE International Conference on Teaching, Assessment, and Learning for Engineering.
  91. The value of a smile: Game theory with a human face. Journal of Economic Psychology 22, 5 (2001), 617–640.
  92. Science of People. 2022. The Ultimate Guide to Body Language. https://www.scienceofpeople.com/category/body-language/
  93. Robots in groups and teams: a literature review. Proceedings of the ACM on Human-Computer Interaction 4, CSCW2 (2020), 1–36.
  94. Atsushi Senju and Mark H Johnson. 2009. The eye contact effect: mechanisms and development. Trends in cognitive sciences 13, 3 (2009).
  95. S. S. SHAPIRO and M. B. WILK. 1965. An analysis of variance test for normality (complete samples). Biometrika 52, 3-4 (1965).
  96. Where to look: a study of human-robot engagement. In Proceedings of the 9th international conference on Intelligent user interfaces. 78–84.
  97. Beholden: the emotional effects of having eye contact while breaking social norms. Frontiers in psychology 12 (2021), 545268.
  98. Do tablets really support discussion? Comparison between paper, tablet, and laptop PC used as discussion tools. In Proceedings of the 24th Australian Computer-Human Interaction Conference. 562–571.
  99. Niels ter Heijden and Willem-Paul Brinkman. 2011. Design and evaluation of a virtual reality exposure therapy system with automatic free speech interaction. Journal of CyberTherapy and Rehabilitation 4, 1 (2011).
  100. Who Do You Prefer? The Effect of Age, Gender and Role on Users’ First Impressions of Embodied Conversational Agents in eHealth. International Journal of Human–Computer Interaction 36, 9 (2020).
  101. The effect of multimodal emotional expression and agent appearance on trust in human-agent interaction. In Proceedings of the 12th ACM SIGGRAPH Conference on Motion, Interaction and Games. 1–6.
  102. If your device could smile: People trust happy-sounding artificial agents more. Computers in Human Behavior 105 (2020), 106215.
  103. Identity and epistemic emotions during knowledge revision: A potential account for the backfire effect. Discourse Processes 53 (2016).
  104. A.J. Unterweger. 2022. Emotional involvement in group discussions : an explorative study.
  105. Marleen van de Kerkhof. 2006. Making a Difference: On the Constraints of Consensus Building and the Relevance of Deliberation in Stakeholder Dialogues. Policy Sciences 39, 3 (2006), 279–299.
  106. The interpersonal effects of anger and happiness in negotiations. Journal of personality and social psychology 86, 1 (2004), 57.
  107. George Veletsianos. 2012. How do learners respond to pedagogical agents that deliver social-oriented non-task messages? Impact on student learning, perceptions, and experiences. Computers in Human Behavior 28, 1 (2012), 275–283.
  108. HeadBox: A Facial Blendshape Animation Toolkit for the Microsoft Rocketbox Library. In 2022 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW). 39–42. https://doi.org/10.1109/VRW55335.2022.00015
  109. “It doesn’t matter what you are!” Explaining social effects of agents and avatars. Computers in Human Behavior 26, 6 (2010), 1641–1650.
  110. The influence of size in augmented reality telepresence avatars. In 2019 IEEE Conference on Virtual Reality and 3D User Interfaces (VR). IEEE, 538–546.
  111. Exploring virtual agents for augmented reality. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. 1–12.
  112. The Aligned Rank Transform for Nonparametric Factorial Analyses Using Only Anova Procedures. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Vancouver, BC, Canada). 143–146.
  113. Comparing Visual Attention with Leading and Following Virtual Agents in a Collaborative Perception-Action Task in VR. In 2023 IEEE Conference Virtual Reality and 3D User Interfaces (VR). IEEE, 152–162.
  114. Stephen Xavier. 2005. Are you at the top of your game? Checklist for effective leaders. Journal of Business Strategy 26, 3 (2005), 35–42.
  115. Gary Yukl. 1989. Managerial leadership: A review of theory and research. Journal of management 15, 2 (1989), 251–289.
  116. Modelling multi-party interactions among virtual characters, robots, and humans. Presence: Teleoperators and Virtual Environments 23, 2 (2014).
  117. Tracking and fusion for multiparty interaction with a virtual character and a social robot. In SIGGRAPH Asia 2014 Autonomous Virtual Humans and Social Robot for Telepresence. 1–7.
  118. Janice Zeman and Judy Garber. 1996. Display rules for anger, sadness, and pain: It depends on who is watching. Child development 67, 3 (1996), 957–973.
  119. Emotional intelligence, psychological safety, and team decision making. Team Performance Management: An International Journal (2020).
Citations (2)
View on Semantic Scholar

Summary

No one has generated a summary of this paper yet.

Sign Up to Summarize

Paper to Video (Beta)

No one has generated a video about this paper yet.

Sign Up to Generate All Videos Subscribe on YouTube

Whiteboard

No one has generated a whiteboard explanation for this paper yet.

Sign Up to Generate

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Generate Now

Continue Learning

We haven't generated follow-up questions for this paper yet.

Generate Now

Collections

Sign up for free to add this paper to one or more collections.

Sign Up

Tweets

Sign up for free to view the 1 tweet with 0 likes about this paper.

Sign Up for Free