Stress-testing Cross-Cancer Generalizability

• Stress-testing cross-cancer generalizability is a systematic evaluation of ML models' ability to perform on unseen or underrepresented cancer types and conditions.
• It employs rigorous experimental designs, such as cold splits and cross-domain validations, to quantify out-of-distribution performance using metrics like RMSE, C-index, and DSC.
• Methodological strategies like transfer learning, self-attention, and domain adversarial adaptation are key to mitigating OOD degradation and ensuring clinically relevant performance.

Stress-testing cross-cancer generalizability refers to the systematic evaluation of whether machine learning and deep learning models—trained for diagnostic, predictive, prognostic, or therapeutic tasks in oncology—can robustly perform on cancer types, molecular subtypes, institutions, populations, or clinical scenarios that are not represented or seen during model training. This concept encompasses both methodological rigor in experimental design and architectural innovations to explicitly improve, quantify, and explain model performance “out-of-distribution” (OOD) across tumor type, data domain, geography, or protocol. Cross-cancer generalizability has emerged as a focal challenge because models validated via within-cohort or single-institution data frequently display significant degradation under clinically realistic deployment involving new cancer types or centers.

1. Formalizing Cross-Cancer Generalizability

Modeling cross-cancer generalizability typically requires clear, rigorous formalization of training and test domains, data modalities, and task objectives. Problem definition involves structuring datasets into {source, target} pairs, where the target domain includes unobserved or underrepresented cancer types, drugs, or patient features. For example, in machine learning-based drug response prediction, a dataset $\mathcal{D} = \{ (d_j, c_i, y_{ij}) \}$ comprises drug $d_j$ (with multi-modal descriptors), cell line $c_i$ (with omics), and label $y_{ij}$ (IC$_{50}$ or sensitivity). The generalization challenge is to learn a function $f_\theta$ such that $f_\theta(d_j, c_i) \approx y_{ij}$ remains accurate when applied to $(d_j, c_i)$ pairs where $d_j$ (drug scaffold), $c_i$ (cell cluster), or both were unseen during training (Xia et al., 2023). In prognosis or segmentation, generalizability may denote the ability of multimodal or unimodal models trained on one or several cancer types to perform robustly on distinct and potentially biologically dissimilar types or clinical cohorts (Zhou et al., 16 Sep 2025, Jiang et al., 11 Jul 2025, Ghosh et al., 26 Aug 2025).

2. Stress-Testing Protocols and Experimental Designs

Stress-testing is operationalized via systematic “cold start” and domain-shift experimental setups, which may include:

• Cold Drug/Scaffold Split: Entire drugs or molecular scaffolds withheld from training, evaluating predictions on structurally novel agents (Xia et al., 2023).
• Cold Cell/Cluster Split: Cell lines, samples, or clusters (by transcriptomic or mutational profile) wholly held out, simulating unseen tumor biology (Xia et al., 2023, Jiang et al., 11 Jul 2025).
• Combined Cold Split: Simultaneous withholding of drugs and cell clusters for the strictest OOD assessment.
• Cross-domain Evaluation: Train on one or more cancer domains, test on others (e.g., train on oesophageal PET–CT, test on lung PET–CT) (Ghosh et al., 26 Aug 2025).
• Pan-cancer or Single-domain Generalization: Train on one cancer type, test on several others with distinct distributions (Jiang et al., 11 Jul 2025).
• External Cohort Validation: Pre-train on large, multi-cancer cohorts (e.g., TCGA), test on truly independent datasets annotated with outcomes (e.g., HANCOCK, YNGC, NFCRC) (Zhou et al., 16 Sep 2025).

Evaluation metrics are selected to match clinical and technical objectives, e.g., RMSE, C-index, Dice similarity coefficient, area under ROC/PR for classification, and protocol-specific quantifications such as 3D Gamma passing rate in dose prediction (Xia et al., 2023, Zhang et al., 2023, Ghosh et al., 26 Aug 2025). These setups are frequently realized as $k$-fold cross-validation or leave-one-cohort/class-out paradigms, and accompanied by rigorous baseline comparisons and ablations.

3. Architectural and Methodological Strategies

Several methodological innovations target cross-cancer generalizability by explicitly mitigating OOD degradation:

• Transfer Learning and Foundation Models: Pre-training molecular, imaging, or language encoders on large-scale, out-of-domain data (e.g., ChemBERTa on 10M SMILES (Xia et al., 2023); pathology foundation models in MICE (Zhou et al., 16 Sep 2025)).
• Self-Attention and Multimodal Fusion: Deploying attention-based fusion modules to integrate diverse data modalities (fingerprints, pathways, WSI, omics), allowing for dynamic reweighting and information exchange (Xia et al., 2023, Zhou et al., 16 Sep 2025, Jiang et al., 11 Jul 2025).
• Collaborative Expert and Modular Architectures: Models like MICE use consensual, cancer-specific (specialized), and overlapping Transformer experts, coupled with routers to balance global knowledge and domain specialization (Zhou et al., 16 Sep 2025).
• Domain Adversarial Adaptation: Application of DANNs (Domain-Adversarial Neural Networks) to align feature representations across tissue- or organ-specific histopathology domains, demonstrably lifting zero-shot accuracy from random to clinically relevant levels (Cheung et al., 21 Jan 2026).
• Physics-informed Inputs: In radiotherapy dose prediction, encoding physical constraints and distributions via low-statistics Monte-Carlo “noisy probing dose” channels yields generalizable models robust to previously unseen treatment geometries (Zhang et al., 2023).
• Sparse Dirac Rebalancer and Distribution Entanglement: Plugins such as SDIR enforce active fusion from weaker modalities by Bernoulli channel dropout and Dirac stabilization; CADE regularizes latent distributions to mimic unseen cancer types (Jiang et al., 11 Jul 2025).

4. Benchmarking, Ablation, and Quantitative Results

Robust stress-testing is realized by benchmarking new methods against strong baselines (e.g., DeepCDR, DeepTTA, GraphDRP, multi-head/ensemble multimodal models) and performing extensive ablation to clarify the contributions of each component.

Selected quantitative highlights:

Scenario	Metric	Best Achieved	Comparator(s)	Reference
Cold cell&scaffold (CDR)	Pearson’s $r$	0.4146	0.30–0.38 (DeepCDR, TGSA, etc.)	(Xia et al., 2023)
Cross-domain segmentation	DSC (combined)	52.9 (LC)	1.3 (OC-only), 51.6 (AutoPET-only, LC)	(Ghosh et al., 26 Aug 2025)
Pan-cancer prognosis	C-index (MICE)	0.710	0.634–0.672 (MoE, Multi-head, etc.)	(Zhou et al., 16 Sep 2025)
Multimodal cross-cancer SDG	C-index (SDIR+CADE)	0.5625	0.5369 (DFQ, SOTA multimodal)	(Jiang et al., 11 Jul 2025)
Histopathology OOD	Accuracy (DANN)	95.6 (lung)	52.3 (ResNet50, supervised only)	(Cheung et al., 21 Jan 2026)
Physics-aware dose pred.	Gamma Pass (%)	≥96.8 (targets, Exp3)	89.3–93.4 (Exp1–Exp2, prostate)	(Zhang et al., 2023)

Further, ablation studies in these works reveal that omitting transfer learning, self-attention, expert specialization, or physics-informed features sharply reduces generalizability. For example, TransCDR’s pre-trained encoders delivered an absolute Pearson’s $r$ boost of ≥0.04 over RNN-from-scratch in the “warm start”, and the absence of SDIR and CADE in multimodal prognosis models consistently lowered C-index in all pan-cancer splits (Xia et al., 2023, Jiang et al., 11 Jul 2025).

5. Determinants, Limitations, and Dataset Design

Cross-cancer generalizability is often bottlenecked by:

• Training Data Diversity: Dataset heterogeneity across cancer types, drug/compound space, demographic and scanner sources is typically more impactful than marginal algorithmic innovation (Ghosh et al., 26 Aug 2025, Xia et al., 2021).
• Assay and Imaging Modality: Technical mismatches in viability assays or imaging protocols can cause pronounced OOD degradation, only partially correctable by feature harmonization or domain adaptation (Xia et al., 2021, Cheung et al., 21 Jan 2026).
• Modality Imbalance: Unimodal models may, paradoxically, outperform vanilla multimodal models cross-cancer, as strong modalities dominate fusion unless explicitly balanced (Jiang et al., 11 Jul 2025).
• Limited Pan-Cancer Benchmarks: Many studies feature limited “leave-one-cancer-out” or zero-shot splits; independent multi-institution cohorts are still relatively scarce (Zhou et al., 16 Sep 2025).

A plausible implication is that future progress will require both larger, rigorously annotated pan-cancer repositories spanning multiple technical and biological axes, and methodological advances that explicitly model domain shift and distributional uncertainty.

6. Biological Interpretability and Clinical Implications

Beyond raw predictive accuracy, models stress-tested for cross-cancer generalizability increasingly pursue explanatory or mechanistic validation. For example:

• Gene Set Enrichment Analysis: Stratifying TCGA patients by predicted drug response clusters and performing GSEA recovered known resistance and pathway signatures (e.g., EGFR and EMT for Afatinib) (Xia et al., 2023).
• Feature Attribution: Integrated Gradients for DANN revealed that high-confidence predictions relied on biologically meaningful regions, such as clusters of malignant nuclei, not color artefacts (Cheung et al., 21 Jan 2026).

These findings support not only technical generalizability but also biological plausibility, a prerequisite for regulatory and translational acceptance.

7. Open Problems and Future Directions

Key limitations and desiderata recognized across studies include:

• Strict Zero-Shot Evaluation: More rigorous leave-one-cancer-out or zero-shot setups (where an entire domain/type is unseen throughout training and hyperparameter selection) are needed for realistic stress-testing (Zhou et al., 16 Sep 2025).
• Training on Diverse Modalities: Incorporating new data types (radiomics, spatial -omics, clinical narratives) and dynamically weighting them depending on availability and domain (Zhou et al., 16 Sep 2025, Jiang et al., 11 Jul 2025).
• Adaptive or Learnable Fusion: Sample-adaptive modality fusion and distributional entanglement (beyond fixed Bernoulli dropout or Gaussian constraints) could further attenuate domain shift (Jiang et al., 11 Jul 2025).
• Scalability and Data Efficiency: Foundation models that retain robustness under data scarcity and missing modalities (Zhou et al., 16 Sep 2025).
• Physics Incorporation: Extension of physics-driven strategies (e.g., probing dose) to additional cancer types and multi-institution input (Zhang et al., 2023).

Overall, the field continues to shift toward holistic benchmarks, explicit domain adaptation, and biologically-informed architectures, all guided by systematic and quantitative stress-testing protocols that expose—and ultimately address—the substantial barriers to truly cross-cancer generalizable machine learning in oncology.