Mitigating Selection Bias with Node Pruning and Auxiliary Options

Abstract: LLMs often exhibit systematic preferences for certain answer choices when responding to multiple-choice questions-a behavior known as selection bias. This bias reduces the accuracy and reliability of LLM outputs, limiting their usefulness in decision-critical applications. While prior work has focused on adjusting model inputs or outputs to mitigate this issue, our work takes a fundamentally different approach by identifying and removing the internal sources of bias. We introduce two methods: Bias Node Pruning (BNP), which prunes parameters that contribute to selection bias, and Auxiliary Option Injection (AOI), which introduces an additional answer choice to reduce bias in both white-box and black-box settings. To address the shortcomings of existing evaluation metrics, we propose Choice Kullback-Leibler Divergence (CKLD), a new metric that captures distributional imbalances in model predictions. Experiments on three LLMs across multiple datasets demonstrate that our methods consistently improve answer accuracy while reducing selection bias, providing a robust solution for both open- and closed-source models.

• The paper introduces Bias Node Pruning (BNP) and Auxiliary Option Injection (AOI) to effectively reduce selection bias in multiple-choice questions posed to LLMs.
• Experimental results on ARC-Challenge and other benchmarks demonstrate accuracy improvements from 52.3% to 65.3% and significant CKLD reductions.
• The study offers practical tools for enhancing model reliability while providing new theoretical insights into mitigating internal biases at the parameter level.

Mitigating Selection Bias in LLMs using Node Pruning and Auxiliary Options

The paper "Mitigating Selection Bias with Node Pruning and Auxiliary Options" addresses a critical problem in LLMs: selection bias. Selection bias, the model's unwarranted preference for certain options in multiple-choice questions (MCQs), undermines the reliability of LLMs in tasks requiring unbiased decision-making. This research introduces two novel debiasing techniques—Bias Node Pruning (BNP) and Auxiliary Option Injection (AOI)—to effectively mitigate selection bias and improve MCQ performance.

Main Contributions

Bias Node Pruning (BNP):

BNP identifies and eliminates specific nodes in the LLM's final linear layer contributing to selection bias. This method, driven by its unique approach to scrutinize embedding-level discrepancies, reduces bias without extensively modifying the model's structure. By doing so, BNP leverages the intrinsic properties of the model's parameters, enhancing model reliability and performance.

Auxiliary Option Injection (AOI):

AOI introduces an auxiliary "I don't know" option in the MCQ input, aiming to reduce bias induced by uncertainty. This technique is simple, yet effective, and can be applied to black-box models as well. It ensures the LLM provides responses that better reflect the inherent uncertainty in situations where the model might otherwise exhibit biased preferences.

Evaluation Metrics:

The research critiques existing metrics like Standard Deviation of Recalls (RStd) and Relative Standard Deviation (RSD) for their insensitivity to label imbalance. It introduces Choice Kullback-Leibler Divergence (CKLD) as a novel metric sensitive to label distribution imbalance, providing a more rigorous assessment of selection bias.

Experimental Results

The proposed methods were evaluated on various datasets including ARC-Challenge, MMLU-Redux, and CommonsenseQA using different LLMs like Llama-3, Mistral, and Bloomz. The results demonstrated significant improvements in both bias mitigation and performance metrics:

• Llama-3 Performance:
  • BNP and AOI improved accuracy on ARC-Challenge from 52.3% to 65.3%, marking a substantial enhancement of 24.9%.
  • CKLD decreased from 0.494 to 0.124, indicating reduced selection bias.
• Bloomz and Mistral:
  • Both models exhibited performance improvements when BNP and AOI were applied, with AOI showing particularly strong adaptability across black-box scenarios.
  • CKLD values improved significantly in almost all scenarios, reaffirming the effectiveness of the proposed methods.

The experiments further validated the robustness of BNP and AOI through various ablation studies, underscoring their applicability across different model architectures and dataset distributions.

Practical and Theoretical Implications

From a practical standpoint, reducing selection bias enhances the reliability of LLM applications in automated systems, including data annotation and decision-making tasks. The proposed methods not only improve accuracy but also ensure that the model's predictions are more consistent and unbiased.

Theoretically, this research provides new insights into the internal mechanisms leading to selection bias, shifting the focus from superficial bias correction to addressing root causes at the parameter level. This paradigm shift opens new avenues for future research to explore internal model behaviors and their impacts on output biases.

Future Directions

This work paves the way for further research into the internal sources of biases in LLMs. Future studies could investigate the underlying factors contributing to selection bias across different model families and data types. Additionally, the methodologies introduced could be extended to explore other types of biases, providing a holistic approach to improving LLM reliability.

In conclusion, the introduction of Bias Node Pruning and Auxiliary Option Injection represents a significant step towards mitigating selection bias in LLMs. By addressing the problem at the parameter level and incorporating auxiliary options to manage uncertainty, this research offers practical and theoretical advancements essential for the development of unbiased, reliable AI systems.