Exploring Big Five Personality and AI Capability Effects in LLM-Simulated Negotiation Dialogues

Abstract: This paper presents an evaluation framework for agentic AI systems in mission-critical negotiation contexts, addressing the need for AI agents that can adapt to diverse human operators and stakeholders. Using Sotopia as a simulation testbed, we present two experiments that systematically evaluated how personality traits and AI agent characteristics influence LLM-simulated social negotiation outcomes--a capability essential for a variety of applications involving cross-team coordination and civil-military interactions. Experiment 1 employs causal discovery methods to measure how personality traits impact price bargaining negotiations, through which we found that Agreeableness and Extraversion significantly affect believability, goal achievement, and knowledge acquisition outcomes. Sociocognitive lexical measures extracted from team communications detected fine-grained differences in agents' empathic communication, moral foundations, and opinion patterns, providing actionable insights for agentic AI systems that must operate reliably in high-stakes operational scenarios. Experiment 2 evaluates human-AI job negotiations by manipulating both simulated human personality and AI system characteristics, specifically transparency, competence, adaptability, demonstrating how AI agent trustworthiness impact mission effectiveness. These findings establish a repeatable evaluation methodology for experimenting with AI agent reliability across diverse operator personalities and human-agent team dynamics, directly supporting operational requirements for reliable AI systems. Our work advances the evaluation of agentic AI workflows by moving beyond standard performance metrics to incorporate social dynamics essential for mission success in complex operations.

• The paper demonstrates that manipulating Big Five personality traits like Agreeableness and Extraversion significantly improves goal achievement and interaction quality in simulated negotiations.
• It employs two experiments using the Sotopia framework to evaluate both interpersonal price negotiation and human-AI job negotiation scenarios.
• The study suggests that integrating adaptive AI traits with human personality insights can enhance trust and efficacy, guiding the design of future AI systems.

Exploring Big Five Personality and AI Capability Effects in LLM-Simulated Negotiation Dialogues

The paper "Exploring Big Five Personality and AI Capability Effects in LLM-Simulated Negotiation Dialogues" (2506.15928) provides an in-depth analysis of how personality traits and AI agent characteristics affect negotiation outcomes in simulated environments. Utilizing the Sotopia simulation framework, the authors conduct two main experiments focusing on interpersonal and human-AI negotiation contexts.

Simulation Framework and Experiments

The framework revolves around creating negotiation scenarios where LLMs simulate human interactions. This provides a rich ground to study the effects of different personality traits as described by the Big Five model: Agreeableness, Conscientiousness, Extraversion, Neuroticism, and Openness.

Experiment 1: Interpersonal Price Negotiation

In the first experiment, personality traits are manipulated in a price bargaining scenario to observe their influence on negotiation outcomes. Here, key metrics include goal achievement and interaction quality.

Figure 1: Sotopia simulation framework.

The results indicate that Agreeableness and Extraversion significantly impact negotiation outcomes such as believability and goal achievement. The positive effects align with established theories linking these traits to enhanced cooperation and assertiveness.

Experiment 2: Human-AI Job Negotiation

The second experiment focuses on job negotiation scenarios involving an AI agent and a human digital twin. By adjusting AI characteristics like transparency and adaptability, the study explores their combined effect with human personality traits.

Figure 2: Trait level--Sotopia-Eval SEM Weights.

Figure 3: Empathy Emotion Measures.

The findings highlight that while AI traits like adaptability positively influence interaction dynamics, human traits such as Extraversion continue to have dominant roles in shaping trust and negotiation efficacy.

Discussion and Implications

The paper provides substantial empirical evidence that personality trait manipulations can effectively simulate expected human negotiation behaviors. For AI agents deployed in mission-critical applications, designing systems that can adapt to diverse operator personalities is crucial for sustaining operational effectiveness.

Future Directions

Future research could explore more complex scenarios with added variables or test the framework in real-world applications. Furthermore, incorporating additional AI characteristics, such as emotional intelligence and decision-making transparency, could enhance the realism and applicability of simulation outcomes.

Conclusion

The paper demonstrates that LLM-based simulations offer a potent tool for examining intricate social dynamics, providing insights that could refine AI deployment strategies in domains ranging from defense to commercial negotiations. By bridging the gap between theoretical constructs and practical applications, it sets a foundation for the continued exploration of AI and human behavioral interactions.