Generalizability Paradox in Research

• Generalizability Paradox is a phenomenon where results validated within a specific experimental context do not extend reliably to different populations or conditions.
• It manifests through biases such as effect amplification, sign reversal, and attribute importance reversal, complicating causal inference and predictive accuracy.
• Resolving the paradox requires careful experimental design, diverse sampling, and balancing in-sample precision with external validity across varied real-world domains.

The generalizability paradox captures a recurrent and foundational phenomenon across experimental sciences and machine learning, wherein empirical findings—though internally valid and rigorously derived—fail to reliably extend beyond the specific contexts, conditions, or samples from which they were obtained. This paradox reflects a deep discord between conventional estimands of causal or predictive accuracy and the elusive goal of external validity: ensuring that study conclusions persist or replicate across varying backgrounds, populations, or experimental regimes. The paradox manifests in a range of applications, including social science experiments, supervised learning, and adaptive data analysis, where differing sources of bias, sample coverage, and design limitations can yield systematically conflicting or non-portable results.

1. Formal Models and Definitions

Generalizability, in its broadest technical sense, denotes the capacity for an inference or result to remain invariant across changes in key experimental, environmental, or population-level factors. Formally, in the multiconditional experimental paradigm of (Matteucci et al., 2024), a research question is defined as $Q=(A, C, I_{\mathrm{var}}, \mathrm{goals})$, where $A$ denotes alternatives (e.g., candidate models), $C$ is the set of experimental conditions, $I_{\mathrm{var}}$ specifies the subset of factors varied to assess portability, and outcomes are cast as distributions over results (e.g., rankings or effect sizes). The generalizability metric then quantifies the likelihood that empirical distributions, obtained from independent samples of size $n$, are within a pre-specified metric distance ($\varepsilon$) with high probability ($\alpha$), often measured by Maximum Mean Discrepancy (MMD) on the space of results.

In multidimensional causal inference—exemplified by conjoint analysis—the formal model (Fu et al., 2024) represents decision-maker utility as

$V_i(A) = \sum_{k=1}^K \alpha_k u_k(X_k, \theta_i)$

where $\alpha_k$ encodes attribute salience and $u_k$ is an attribute-specific utility function. Experimental average marginal component effects (AMCEs) and individual treatment effects (ITEs) are functions of the salience allocation and background attribute distribution, embedding the dependence of empirical findings on high-dimensional context.

In causal effect estimation, the combinatorial external validity framework (Ribeiro, 2021) proposes that the externally valid effect for treatment $a$ is

$$\mathrm{EV}(a) = \mathrm{Var}^{-1}[ \Delta y(a) \mid \Pi_n(B)]$$

where $\Pi_n(B)$ denotes the ensemble of observed "backgrounds" (settings/assignments of non-treatment factors), emphasizing variance across backgrounds as the operational criterion for generalizability.

2. Amplification, Sign Reversal, and Attribute Importance Reversal

A central manifestation of the generalizability paradox arises from the interplay of limited attention, attribute salience, and context variability in multidimensional experiments (Fu et al., 2024). Three nontrivial pathways by which experimental effects diverge from real-world effects have been identified:

• Effect amplification (Amplification Bias): When experiments omit salient real-world attributes (limited attention), the experimental ITE is inflated relative to the real-world ITE:

$$ITE^{\mathrm{exp}} = \frac{1}{\sum_{j=1}^k \alpha_j} ITE^{\mathrm{real}}, \qquad \delta > 1$$

Empirically, as the number of attributes increases, observed AMCEs decrease (meta-analyses in political candidate conjoint studies and hotel room choice experiments).

• Sign reversal/null attenuation: Salience effects can alter the rank order or dominance of attributes across profiles. If a treatment manipulates an attribute sufficiently to change psychological salience ordering, observed effects can flip sign (or attenuate to zero) relative to real-world effects. Empirical data show sign reversals in causal effect estimates as attribute number increases, which can be mitigated by constraining treatment levels to preserve salience ranks.
• Attribute importance reversal: The ordering of attributes (by AMCE magnitude or explanatory R-squared) in the experiment may not reflect their real-world importance due to context-dependent salience reweighting, producing instability in interpretive conclusions.

These findings are supported by both formal propositions and large-scale empirical studies, emphasizing the necessity of realistic attribute selection and salience calibration for external validity in multidimensional designs (Fu et al., 2024).

3. Combinatorial and Randomization-Based Limits

The combinatorial approach formulated in (Ribeiro, 2021) demonstrates two distinct regimes where in-sample unbiasedness is decoupled from external validity:

• Enumeration Regime: External validity is achievable if all possible "backgrounds" (covariate assignments) appear at least once in the sample—thus, observed effects represent a complete tour of context permutations.
• Randomization Regime: If observed backgrounds are drawn uniformly at random, background dependencies are sufficiently broken, and effect variance across backgrounds collapses with large sample size.

However, standard causal inference methodologies focus primarily on bias reduction within fixed backgrounds, lacking guarantees when unobserved background variation or structure is present, thus instantiating the generalizability paradox: precise in-sample estimation does not entail generalizability unless sample backgrounds are either fully enumerated or randomization is demonstrably adequate. Bias–variance–external validity tradeoffs must therefore be explicitly addressed in both the design phase and interpretive analysis (Ribeiro, 2021).

4. Sample Complexity and Contradictory Benchmarks

Generalizability in machine learning encompasses the stability of empirical conclusions across datasets, tasks, or experimental factors. The framework in (Matteucci et al., 2024) formalizes the sample complexity required to certificate generalizability. For given factors allowed to vary and a similarity metric ($\varepsilon$), the number of samples ($n^*$) needed for empirical results to stabilize is given by

$n^* = \exp(\beta_1 \log \varepsilon^* + \beta_0)$

where $(\beta_0, \beta_1)$ are empirically fitted via pilot studies for a chosen generalizability ($\alpha^*$).

Contradictory conclusions across benchmarks—such as observed in state-of-the-art ML comparison studies—are not paradoxical but rather a consequence of insufficient sample coverage:

• Differing choices of varied factors ($I_{\mathrm{var}}$), metrics (kernels on result spaces), or low sample sizes ($n < n^*$) allow empirical frequencies to fluctuate, yielding materially conflicting recommendations even with reproducible protocols.
• The "paradox" disappears when explicit attention is paid to the dimensions over which generalizability is desired, and sample sizes are matched to required stability thresholds (Matteucci et al., 2024).

5. Adaptive Data Analysis and Irreducible Barriers

Adaptive data analysis highlights an intrinsic generalizability paradox resulting from feedback-driven exploration. Mechanisms that ensure post-selection generalization—i.e., that limit the adversarial discovery of queries with large empirical–population gaps—must inject noise of order $\Omega(\sqrt{k/n})$ for $k$ adaptive queries over $n$ samples (Nissim et al., 2018). Composition of such mechanisms, even if each is robust in isolation, fails in aggregate: releasing outputs from multiple post-selection-safe algorithms can reconstruct the original data and defeat the generalization guarantee altogether.

Thus, there exists a sharp trade-off:

• Sound adaptive generalization is only possible at the price of reduced accuracy (noise scale $\sqrt{k/n}$) and cannot be modularly extended by composing multiple generalization-safe procedures.
• The paradox is that methods designed to robustly protect generalizability in the post-adaptive regime either become too noisy to be useful or collapse under multi-stage data re-use (Nissim et al., 2018).

6. Data Geometry, Dimensionality, and Weak Predictors of Portability

Systematic meta-analyses in machine learning reveal that commonly invoked notions of "curse of dimensionality" are not predictive of out-of-sample generalization in real data. In a large-scale study (Barbiero et al., 2020), intrinsic dimensionality and related manifold metrics displayed almost zero correlation with generalization performance ($F_1$) on held-out data:

• Model performance on test data and especially on points outside the convex hull of the training set is better predicted by featurewise correlation, variance structure, and local statistics rather than global dimensions.
• Generalizability for interpolation (test points inside the convex hull) is more predictable from dataset statistics than for extrapolation, reflecting inherent limits on porting machine learning conclusions to novel regimes. These findings suggest that generalizability paradoxes in ML are less about sparsity or high-dimensionality per se and more about unmeasured structure, background coverage, and sample representativeness (Barbiero et al., 2020).

7. Resolving and Mitigating the Generalizability Paradox

Collectively, these lines of work establish theoretical, methodological, and pragmatic implications:

• Design recommendations for experiments: Restrict attribute space to realistic, context-matched choices (to avoid artificial salience), and highlight robust dimensions central to real-world decisions (Fu et al., 2024).
• Quantifying required sample diversity: Explicitly compute or estimate the number of independent experimental conditions or sample size necessary to ensure stable, externally valid results given specified generalizability thresholds (Matteucci et al., 2024).
• Bias-variance-external validity tradeoffs: Acknowledge that optimizing for in-sample precision or low variance may decrease generalizability, unless extensive randomization or background enumeration is employed (Ribeiro, 2021).
• Limits of adaptivity: Recognize insurmountable accuracy losses in robust adaptive inference; modular aggregation of analysis tools nullifies guarantees (Nissim et al., 2018).
• Empirical cautiousness: Avoid overinterpretation of benchmark or empirical study conflicts as mysteries; most stem from incomplete coverage of intended generalization dimensions and misalignment between experimental design and inferential scope.

The generalizability paradox ultimately enforces a rigorous perspective on how scientific and predictive inferences should be validated, interpreted, and translated across varying contexts, highlighting the necessity for explicit modeling of sample coverage, attribute salience, randomization, and the design space relevant to the generalization target.