OpenBias: Open-set Bias Detection in Text-to-Image Generative Models

Abstract: Text-to-image generative models are becoming increasingly popular and accessible to the general public. As these models see large-scale deployments, it is necessary to deeply investigate their safety and fairness to not disseminate and perpetuate any kind of biases. However, existing works focus on detecting closed sets of biases defined a priori, limiting the studies to well-known concepts. In this paper, we tackle the challenge of open-set bias detection in text-to-image generative models presenting OpenBias, a new pipeline that identifies and quantifies the severity of biases agnostically, without access to any precompiled set. OpenBias has three stages. In the first phase, we leverage a LLM to propose biases given a set of captions. Secondly, the target generative model produces images using the same set of captions. Lastly, a Vision Question Answering model recognizes the presence and extent of the previously proposed biases. We study the behavior of Stable Diffusion 1.5, 2, and XL emphasizing new biases, never investigated before. Via quantitative experiments, we demonstrate that OpenBias agrees with current closed-set bias detection methods and human judgement.

• The paper introduces a novel open-set framework that detects biases beyond predefined categories by leveraging LLMs and VQA models.
• It employs a three-stage pipeline—bias proposal, image generation, and bias assessment—to quantify biases in models like Stable Diffusion.
• Quantitative evaluations align with human judgment and reveal significant variations in bias across different generative model configurations.

OpenBias: Open-set Bias Detection in Text-to-Image Generative Models

The paper "OpenBias: Open-set Bias Detection in Text-to-Image Generative Models" addresses the significant challenge of identifying biases in text-to-image (T2I) generative models without relying on predefined sets of bias categories. This research is particularly relevant as T2I models see increasing deployment and more widespread use, necessitating an investigation into potential biases these systems might inadvertently amplify or propagate.

The authors introduce a novel framework, OpenBias, which operates in an open-set scenario, aiming to detect and quantify biases in T2I models such as Stable Diffusion, including its versions 1.5, 2, and XL. Unlike prior efforts that often focus on predefined sets of biases, OpenBias seeks to uncover biases potentially overlooked in prior studies.

Methodology Overview

OpenBias comprises a three-stage pipeline leveraging LLMs and Vision Question Answering (VQA) models. The key stages are:

1. Bias Proposal: Utilizing a LLM, OpenBias first proposes potential biases using a set of real-world captions. By generating bias proposals without any precompiled list, the framework taps into vast uncharted domains of biases beyond typical categories such as gender or race.
2. Image Generation: For each proposed bias, the associated generative model processes the same set of captions to produce images. This stage serves as a placeholder for generating visual representations that might harbor the proposed biases.
3. Bias Assessment: A VQA model evaluates the generated images to recognize and quantify the presence of the candidate biases. This modeling of bias assessment enables OpenBias to effectively capture the subtleties in bias propagation in a contextual and context-free manner.

Key Results

The paper contains robust numerical evaluations that substantiate the framework's claims. Quantitative experiments exhibit remarkable agreement between OpenBias and conventional closed-set bias detection methodologies as well as with human judgment, validating the framework's reliability. The research also highlights differences in bias prevalence across different configurations of the Stable Diffusion model, demonstrating the nuanced discoveries that an open-set approach enables.

Theoretical and Practical Implications

Theoretically, this research extends our understanding of bias detection in machine learning models by challenging the notion that biases must be pre-specified. It proposes that there are often naïve assumptions embedded in datasets which may lead to unchecked bias propagation. Practically, OpenBias opens the door for AI developers and researchers to identify bias in a more comprehensive fashion, potentially improving the fairness of model outputs in real-world applications.

Future Speculations

Moving forward, the framework's adaptability presents various opportunities for further exploration. Several next steps can include deploying OpenBias to critically assess other forms of generative AI, such as language generation models, and extending it to include audio-visual and multimodal systems. Additionally, integrating OpenBias into bias mitigation protocols could further enhance the fairness of T2I models.

In summary, OpenBias provides an invaluable tool for broadening the scope of bias detection in T2I models, advocating for a shift towards a more dynamic and context-sensitive understanding of bias. This research represents a crucial advancement in the ongoing efforts to foster more equitable machine learning systems.