Bias Sinks: Unintended Amplification of Biases

• Bias sinks are structural or algorithmic features that absorb and amplify pre-existing biases, creating fairness and performance challenges across multiple domains.
• In machine learning, bias sinks emerge in embedding spaces and evaluation models, skewing outputs and undermining fairness despite balanced data.
• In decision processes and materials science, bias sinks trap agents in suboptimal policies and affect defect absorption, highlighting the need for targeted countermeasures.

A bias sink is a structural, algorithmic, or statistical feature within a system that absorbs, amplifies, or reinforces pre-existing biases, preventing unbiased operation or evaluation. The term arises in diverse domains including decision theory, machine learning (especially generative models), and radiation materials science. Across these areas, bias sinks are characterized by their ability to transform local or latent asymmetries into globally observable or operationally consequential skew, often undermining intended fairness, efficiency, or resilience properties.

1. Formal Definitions and Theoretical Foundations

Bias sinks manifest when a subsystem or mechanism creates a one-way trap that amplifies or absorbs bias. In text-to-image (TTI) diffusion pipelines, a bias sink is typically an embedding space (either input text encoders or evaluation models like CLIP) in which representational asymmetries are hidden or intensified, absorbing otherwise correctable unfairness into generation or evaluation out-puts (Kuchlous et al., 2024). In stochastic planning with sunk-cost bias, a bias sink describes a dynamic state where accumulated sunk cost creates a terminal 'trap': the agent is incentivized to stop or fails to continue due to the penalty associated with prior investment, thus locking itself into suboptimal policy choices (Kleinberg et al., 2021).

The mathematical formalization depends on context:

• In TTI, let $e_P$ be the prompt encoder and $e_I$ the image encoder. If $\cos(e_P(b), e_P(a+b))$ varies significantly across group attributes $a$, any subsequent generation or evaluation will reflect this skew, regardless of generative intent.
• In Markov decision processes with sunk-cost bias, bias sinks emerge through abandonment penalties $\lambda C_{\rm sunk}$, shifting Bellman-optimal policies (see section 3).

2. Bias Sinks in Machine Learning: Embedding Spaces and Evaluation

Diffusion models use text encoders (e.g., CLIP, W2VNEWS) to condition on prompts and subsequently generate images. Embedding biases arise when attribute-conditioned embeddings (e.g., for gender or profession) are unequally separated in latent space. Theorem 3.1 in (Kuchlous et al., 2024) rigorously establishes that, under mild regularity assumptions on the diffusion score network, an unbiased embedding is necessary for representational fairness. For protected group $G=\{a_1,\ldots,a_k\}$, if $y_b$ (the base embedding) is much closer to $y_{a_1+b}$ than others, the generator will overwhelmingly produce group $a_1$ outputs. Thus, the embedding layer acts as a bias sink, absorbing bias from the encoder and transforming it into output distribution skew—potentially even when the data is balanced.

Bias sinks also affect evaluation: multimodal embeddings like CLIP used for scoring (CLIPScore) will reward outputs closer to their own biased internal representations. As a result, a model that outputs fair, balanced images may register lower alignment scores than an unfair, biased model—a consequence of the bias sink effect of the evaluation pipeline (Kuchlous et al., 2024).

3. Bias Sinks in Stochastic Decision Processes: The Sunk-Cost Model

In the context of agents with sunk-cost bias, bias sinks appear as subdomains of the decision process from which rational escape is inhibited by the penalty for abandoning. The formal model uses a directed acyclic graph (DAG) $G=(V, E)$ representing states and actions. The agent’s value function $V(s,C)$, where $C$ is the current sunk cost, includes the abandonment penalty $-\lambda C$. Once an agent has accumulated nontrivial sunk cost, the perceived extra cost to abandon can outweigh the benefit of optimal continuation, trapping the agent in a loop of suboptimal actions or inaction (Kleinberg et al., 2021).

Sophisticated agents anticipate their future selves will continue to be biased and thus solve the full bias-augmented Bellman equation. While naive agents repeatedly replan under the false assumption of future optimality, both can become trapped by bias sinks—regions of the state–cost space where abandonment costs outweigh potential rewards, even when globally better alternatives exist.

4. Bias Sinks in Materials Science: Interfacial Defect Absorption

In radiation-tolerant materials, a bias sink is a type of interface that does not preferentially absorb vacancies versus interstitials, achieving a near-zero sink bias

$B = (S_i - S_v)/(S_i + S_v),$

where $S_i$ and $S_v$ are the sink strengths for interstitials and vacancies, respectively. Amorphous intergranular films (AIFs) act as near-ideal, unbiased sinks (bias factor $B_{AIF} \simeq 0$) by virtue of their increased thickness ($\sim$2.1 nm) and excess free volume ($\sim$10%) (Ludy et al., 2015). By contrast, ordered grain boundaries demonstrate a significant positive bias ($B_{ord} \sim +0.3$ to $+0.6$), preferentially removing interstitials and failing to absorb vacancies efficiently.

Bias sinks in this context are essential for preventing post-irradiation damage such as void swelling and irradiation-induced creep (from unabsorbed vacancies) or dislocation loop formation (from interstitial excess). Atomistic simulations show that AIFs efficiently annihilate both types of point defects, acting as ultra-efficient, unbiased defect sinks.

5. Quantitative Metrics and Empirical Observations

Machine Learning Pipelines (Kuchlous et al., 2024)

• Statistical group fairness: The model is $v$-representationally-balanced if for all protected groups $i$, $\mathrm{TV}(p_b, p_{a_i+b}) \leq 1-v_i$, where $p_{a_i+b}$ is the output distribution conditioned on attribute $a_i$.
• Empirically, even with perfectly balanced training data, generation outputs skew toward the direction of embedding bias (e.g., "nurse" prompt yielding 59.6% female outputs).
• CLIPScore auditing bias: For images of "doctor," CLIPScore is higher for male versus female representations; models outputting more "male doctor" images are thus unfairly rewarded, creating an evaluation bias sink.

Radiation Materials (Ludy et al., 2015)

• Sink absorption data for AIFs and ordered grain boundaries consistently demonstrate $B_{AIF} \approx 0$ across PKA energies (700–2500 eV), while $B_{ord}$ ranges from 0.3 to 0.6.
• The combined action of interface thickness and free volume underpin the unbiased absorption profile, confirmed by Voronoi analysis and residual defect counting in MD simulations.

Decision Processes (Kleinberg et al., 2021)

• The performance gap for sophisticated agents with bias versus optimal agents is bounded by

$\pi_s \geq \pi_o - \frac{\lambda}{1+\lambda}R,$

where $R$ is the reward at terminal state and $\lambda$ the bias strength. Tight-examples establish that once a small sunk cost is paid, additional continuation can be dominated by the abandonment penalty, functioning as a bias sink.

6. Mechanisms and Countermeasures

Machine Learning

• Debiasing methods target the embedding space, including fair mapping (e.g., projection removal), subclass-score aggregation (choosing the best-matching attribute-prompt for each generated image), and embedding-mean correction (centroid alignment).
• The necessity of unbiased embeddings is both a theoretical and practical requirement: any nontrivial fairness guarantee on output distributions depends on the (approximate) removal of such bias sinks from the underlying encoder or auditing models.

Radiation Materials

• Engineering amorphous intergranular films (via doping, co-sputtering, or grain-boundary complexion engineering) with sufficient thickness and free volume is the direct pathway to create bias-free defect sinks.
• A rational design approach maximizes interfacial thickness ($\geq$2 nm), increases local free volume, and uniformly disperses AIFs through the material matrix to prevent defect build-up.

Decision Processes

• Awareness of bias sinks in agent planning suggests algorithmic modifications such as explicit debiasing of abandonment criteria or structural design of tasks to avoid policy regions that enable bias-trapping.

7. Significance, Implications, and Synthesis

The concept of a bias sink unifies disparate manifestations of hidden or reinforced bias across scientific fields. In generative modeling, bias sinks in embedding or evaluation layers can render fairness guarantees impossible without direct intervention. In stochastic decision processes, sunk-cost-induced bias sinks structurally entrap agents in suboptimal states via their own past expenditures. In materials science, bias sinks represent the ideal: engineered interfaces capable of absorbing both defect types with equal efficiency, preventing accumulations that undermine material performance.

The comparative analysis across ML and physical systems reveals that bias sinks are not merely artifacts but predictable, quantifiable consequences of system structure. The practical implication is a paradigm wherein the removal or mitigation of bias sinks becomes central to algorithm, architecture, and materials design (Kuchlous et al., 2024, Kleinberg et al., 2021, Ludy et al., 2015).