Distributional Creativity Bonus

• Distributional Creativity Bonus (DCB) is a rigorously defined, distribution-based metric that measures creative departures from typical behavior in generative models.
• It employs group-based subset scanning and the Berk–Jones divergence to statistically identify anomalous activations within model hidden layers.
• DCB enables effective model evaluation and hyperparameter tuning by correlating statistical anomalies with human-judged creativity.

A Distributional Creativity Bonus (DCB) is a rigorously defined, distribution-based surrogate for novelty or creativity in the outputs of generative models. Its primary role is to quantify, incentivize, or select for creative departure from model-typical behavior, using principled statistical or information-theoretic machinery grounded in the distributional characteristics of learned representations or outputs. DCBs are widely employed in deep generative modeling, reinforcement learning, and computational creativity to both characterize and promote outputs that deviate from the expected norm while maintaining coherence and utility (Cintas et al., 2021).

1. Mathematical and Algorithmic Foundations

The core principle of a DCB is to compare a generated sample, group of samples, or agent trajectory to a reference distribution representing "typical" or "^^^^1^^^^" behavior, and to extract a scalar bonus that reflects the extent of anomalous, surprising, or creative divergence.

Group-Based Subset Scanning (GBSS) DCB:

Given a generative model with a hidden layer of $J$ nodes, one collects background activations $A^{H_0}_{zj}$ for $Z$ reference samples and $A_{ij}$ for $M$ test samples. For each test activation, compute empirical $p$-values: $$p_{ij} = \frac{1 + \sum_{z=1}^Z \mathbf{1}[A^{H_0}_{zj} \ge A_{ij}]}{Z+1}$$ These $p$-values should be uniformly distributed under the null hypothesis of typical decoding. The algorithm searches for the submatrix $S^* = X_S \times O_S$ most enriched in small $p$-values, maximizing a nonparametric scan statistic such as the Berk–Jones (BJ) divergence: $$F(S) = \max_{\alpha \in (0,1)} N(S)\,\mathrm{KL}\!\left(\frac{N_\alpha(S)}{N(S)} \bigg\| \alpha\right)$$ where $N(S) = |X_S|\,|O_S|$ and $$N_\alpha(S) = \sum_{(i,j) \in S} \mathbf{1}[p_{ij} \le \alpha]$$.

For a batch of generated samples, the maximal scan score $F^*$ is recorded. The DCB for a new batch is its percentile or tail-probability relative to the reference distribution of scores from non-creative samples: $$\mathrm{CB}_p = 1 - \frac{1}{B}\sum_{b=1}^B \mathbf{1}[F^*_b \le F^*_{\text{new}}]$$ This percentile reflects how far into the tail of the reference distribution the observed activation anomaly lies, operationalizing "creative departure" (Cintas et al., 2021, Cintas et al., 2022).

2. Interpretation and Theoretical Properties

A high DCB indicates a coherent, statistically significant departure from the learned reference distribution, interpreted as a neural signature of creative generation. Key empirical and theoretical properties:

• Strong correlation with human-judged creativity: Images or samples with high DCBs are more likely to be labeled creative by evaluators (e.g., 78% agreement on WikiArt (Cintas et al., 2022)).
• DCB values systematically increase with the proportion or intensity of creative samples in a batch.
• The approach is model-agnostic and does not assume a specific parametric form for activation distributions; it employs nonparametric statistical testing.
• Subset scan statistics such as Berk–Jones offer strong power for detecting rare and weak deviations in high-dimensional settings.

Pragmatically, DCBs provide continuous, automatically calibrated metrics suitable for hyperparameter sweeping, architecture selection, or as gating criteria for further human evaluation.

3. Integration and Workflow

The DCB calculation follows a structured workflow:

1. Activation Extraction: For new samples (single or mini-batch), extract hidden-layer activations $\{A_{ij}\}$.
2. Reference Construction: Assemble a background (reference) set of activations $\{A^{H_0}_{zj}\}$ from typical (non-creative) generations.
3. p-Value Calculation: Compute per-node empirical $p$-values, comparing $A_{ij}$ to the background.
4. Scan Statistic Maximization: Using methods such as Linear–Time Subset Scanning (LTSS), efficiently search for subset(s) of nodes and samples exhibiting significant collective deviation.
5. Score Calibration: Convert the maximal scan score $F^*$ of the new sample(s) to a percentile or tail-probability relative to the empirical reference distribution, yielding the DCB.
6. Interpretation: DCB values near 1 (e.g., $\mathrm{CB}_p \geq 0.95$) denote highly creative departures; lower values suggest typicality.

Pseudocode for per-sample bonus computation is provided in (Cintas et al., 2022), with reference CDF construction and scan steps detailed.

4. Empirical Metrics and Validation

Quantitative metrics reported for GBSS-based DCBs include:

Sample Type	Mean $F^*$ (μ)	Std (σ)	Mean Subset Size $|S^*|$	AUROC (Activation Space)
Typical Decoding	5.2	1.1	10	Up to 0.99
Non-Creative Novel	12.4	3.0	50	AUC $>$0.96 (@10% recall)
Creative Decoding	27.8	5.5	200

• A threshold $\mathrm{CB}_p \le 0.05$ (95th percentile) achieves 90% recall with 10% false alarms for creative samples.
• As the prevalence of creative samples decreases in a batch (from 50% to 10%), detection power drops but remains robust ($\mathrm{AUC} > 0.96$).
• The subset size $|S^*|$ indicates the extent of activation-space deviation, larger for creative than typical samples.

These statistics demonstrate the discriminative capacity of DCBs for creativity quantification in activation space, outperforming pixel-level approaches (Cintas et al., 2021, Cintas et al., 2022).

5. Practical Considerations and Applications

Key implementation and application aspects include:

• Layer Selection:

Middle-to-late hidden layers often provide the most discriminative activations for creativity detection.

• Reference Set Size ($Z$):

Larger reference sets yield more precise $p$-values but increase computational and memory cost (values of $Z=250$ demonstrated effective).

• Significance Thresholds:

Only the unique sorted $p$-values need to be considered as significance levels for scan statistics due to the LTSS property.

• Calibration:

Empirical CDFs or parametric tail fits (e.g., generalized Pareto distribution) can be used for robust bonus calibration.

Use Cases:

• Continuous creativity gating in generative model hyperparameter optimization.
• Filtering of candidate generative samples before human curation or evaluation.
• Comparative benchmarking across different model architectures (e.g., VAE, "creative decoders").
• Model-agnostic adaptation to new architectures without retraining or parametric assumptions.

6. Scope, Generalization, and Future Directions

The DCB methodology, as realized through group-based subset scanning, operationalizes a rigorous, distributional definition of creativity grounded in statistical anomaly detection in activation space. Notably:

• It does not assume fixed output spaces or semantic labels, functioning purely on internal representation statistics.
• The approach generalizes naturally to various generative paradigms, including VAEs, GANs, and novel architectures for computational creativity research.
• Significantly, DCBs correlate with human-assigned creativity scores, empirically linking statistical anomaly to perception.
• Extensions to joint sample-node scanning, multi-layer scans, or adaptation to other modalities (text, structured data) are plausible but require further empirical investigation.

This consolidation strictly reflects content from (Cintas et al., 2021) and (Cintas et al., 2022). Any broader conceptual extrapolation is out of scope.