Benchmarking Bias in Large Language Models during Role-Playing

Abstract: LLMs have become foundational in modern language-driven applications, profoundly influencing daily life. A critical technique in leveraging their potential is role-playing, where LLMs simulate diverse roles to enhance their real-world utility. However, while research has highlighted the presence of social biases in LLM outputs, it remains unclear whether and to what extent these biases emerge during role-playing scenarios. In this paper, we introduce BiasLens, a fairness testing framework designed to systematically expose biases in LLMs during role-playing. Our approach uses LLMs to generate 550 social roles across a comprehensive set of 11 demographic attributes, producing 33,000 role-specific questions targeting various forms of bias. These questions, spanning Yes/No, multiple-choice, and open-ended formats, are designed to prompt LLMs to adopt specific roles and respond accordingly. We employ a combination of rule-based and LLM-based strategies to identify biased responses, rigorously validated through human evaluation. Using the generated questions as the benchmark, we conduct extensive evaluations of six advanced LLMs released by OpenAI, Mistral AI, Meta, Alibaba, and DeepSeek. Our benchmark reveals 72,716 biased responses across the studied LLMs, with individual models yielding between 7,754 and 16,963 biased responses, underscoring the prevalence of bias in role-playing contexts. To support future research, we have publicly released the benchmark, along with all scripts and experimental results.

• The paper introduces BiasLens, a framework that uses automated test input and oracle generation to systematically reveal biases in role-playing scenarios.
• It evaluates six advanced LLMs, uncovering 72,716 biased responses and demonstrating that bias levels do not correlate with model performance.
• The study finds that removing role-playing contexts significantly reduces bias, highlighting the role-playing environment as a key factor in bias manifestation.

Analysis of "Benchmarking Bias in LLMs during Role-Playing"

The paper "Benchmarking Bias in LLMs during Role-Playing" addresses a critical aspect of fairness and bias inherent in LLMs when engaged in role-playing activities. LLMs, such as GPT and Llama, have increasingly been deployed in scenarios demanding nuanced human-like interactions, such as finance, law enforcement, and social decision-making. However, these models can reflect and reinforce social biases, especially when simulating diverse roles. The paper introduces BiasLens, a fairness testing framework aiming to expose these biases systematically.

BiasLens Framework

BiasLens incorporates two main components: automatic test input generation and automatic test oracle generation, specifically tailored to uncover biases in LLMs during role-playing.

1. Automatic Test Input Generation: This component uses GPT-4o to generate social roles across 11 demographic attributes, hypothesized to elicit biased behavior due to their potential for discrimination. For each role, BiasLens generates questions designed to trigger biased responses, across Yes/No, Choice, and Why question formats.
2. Automatic Test Oracle Generation: This component distinguishes biased responses using rule-based oracles for Yes/No and Choice questions and an LLM-based oracle for Why questions, validated via a rigorous manual evaluation process.

Evaluation and Results

The study evaluates six advanced LLMs - GPT4o-mini, DeepSeek-v2.5, Qwen1.5-110B, Llama-3-8B, Llama-3-70B, and Mistral-7B-v0.3 - using the BiasLens framework. The evaluation reveals 72,716 biased responses across these models, highlighting the prevalence of bias during role-playing.

1. Impact of Model Capability: Interestingly, the results suggest that biases do not align with model capabilities. Despite being ranked lower in performance, Llama-3-8B shows higher levels of bias than other more capable models. This finding questions the fairness-performance trade-off and suggests the potential for optimizing both dimensions simultaneously.
2. Questions and Role Effect: All question types were effective in triggering biases, with higher prevalence seen in Choice and Why questions. Certain role categories, particularly race and culture, are more susceptible to bias, indicating the potential of reinforcing stereotypical cultural biases within LLMs.
3. Role-Playing Influence: By removing role-playing contexts, the study finds a significant reduction in biased responses, underscoring role-playing as a contributor to increased bias manifestation.

Implications and Future Directions

The findings of this paper have profound implications for both the development and deployment of LLMs in real-world scenarios. The demonstrated biases highlight the pressing need for ongoing fairness testing and mitigation strategies, particularly as LLMs become more integrated into socio-technical systems influencing human decisions and societal outcomes.

Practically, the paper's results emphasize the need for AI developers to incorporate fairness assessments like BiasLens in the deployment pipeline of LLM-based applications. Theoretically, the research prompts further exploration into the mechanisms through which LLMs learn and propagate biases under role-specific conditions. Future research could focus on refining bias detection techniques and developing debiasing algorithms that are robust across diverse roles and applications.

Overall, the paper serves as a crucial contribution to the discourse on fairness in artificial intelligence, providing a comprehensive framework and extensive empirical evaluation that underscores the critical need for fairness in AI, especially in role-specific contexts that are prevalent in real-world applications.