Towards a pretrained deep learning estimator of the Linfoot informational correlation

Abstract: We develop a supervised deep-learning approach to estimate mutual information between two continuous random variables. As labels, we use the Linfoot informational correlation, a transformation of mutual information that has many important properties. Our method is based on ground truth labels for Gaussian and Clayton copulas. We compare our method with estimators based on kernel density, k-nearest neighbours and neural estimators. We show generally lower bias and lower variance. As a proof of principle, future research could look into training the model with a more diverse set of examples from other copulas for which ground truth labels are available.

• The paper introduces a pretrained deep neural network estimator for Linfoot informational correlation that outperforms classical and adaptive methods.
• The study rigorously benchmarks traditional KNN, KDE, and neural MI estimators using Gaussian and Clayton copulas to evaluate estimation bias and variance.
• The application to twin BMI data reveals stronger genetic dependencies than Pearson correlation, underscoring the method's practical impact.

Deep Learning Estimation of the Linfoot Informational Correlation

Introduction

Quantifying statistical dependence between continuous random variables is central to multivariate modeling, yet robust, interpretable, and nonparametric estimators remain elusive especially outside the Gaussian paradigm. The Linfoot informational correlation, a bounded transformation of mutual information, retains desirable properties such as scale invariance, vanishing under independence, and saturation under deterministic relationships—making it a much-needed alternative to the ubiquitous but restrictive Pearson correlation. This paper addresses the practical challenge of accurate Linfoot correlation estimation, introducing and benchmarking a supervised deep learning-based estimator. The performance is rigorously evaluated against classical, nonparametric, and existing neural approaches across various dependence structures, culminating with an application to real-world biomedical twin data.

Theoretical Development: Copula-Based Linfoot Correlation

The authors first provide a functional bridge between the copula parameterization of dependence and the Linfoot informational correlation. The Linfoot correlation, defined as

$L(u, v) = \sqrt{1- \exp(-2I(u;v))}$

where $I(u;v)$ is the mutual information, generalizes the Pearson correlation and satisfies all R\'enyi properties. Critically, for the bivariate Gaussian case $L(u, v) = |\rho|$, recovering the classical result.

Extending this to the broader family of Archimedean copulas, and specifically the Clayton copula, the paper derives a novel closed-form expression for the Linfoot correlation in terms of the Clayton dependence parameter $\theta$. This enables accurate ground-truth benchmarking for non-Gaussian dependency, a prerequisite for training and validating supervised estimators. These analytic developments underscore the mathematical rigor of the approach and provide indispensable calibration points for evaluating estimation bias and variance.

Benchmarking Classical and Neural Mutual Information Estimators

A comprehensive empirical evaluation is performed using both Gaussian and Clayton copulas with varying dependence strengths and sample sizes. The study examines several estimator categories:

• k-Nearest Neighbors (KNN) based estimators: Including standard, truncated, and FNN (digamma-based) algorithms.
• Kernel Density Estimation (KDE) based estimators: Using mirror reflection and Beta kernels to alleviate boundary bias.
• Neural mutual information estimators: Focusing on the MINE approach (Mutual Information Neural Estimation).

These estimators are systematically compared in terms of bias and variance of Linfoot correlation estimates across simulated datasets with known Linfoot values.

Figure 1: Estimation bias for various sample sizes and different copula.

Figure 2: Estimation standard deviation for various sample sizes and different copula.

The FNN estimator provides the best tradeoff among the classical methods but still suffers from both bias and variance issues, especially for strong or weak dependencies and for non-Gaussian copulas. The MINE estimator, while conceptually appealing, exhibits substantial variance and bias—particularly at high dependencies—mirroring known issues with neural mutual information estimators.

Pretrained Deep Neural Network Estimator

The central contribution is a pretrained, supervised deep neural network (DNN) estimator of the Linfoot informational correlation. The training protocol involves simulation of bivariate data from Gaussian and Clayton copulas with fully controlled ground-truth Linfoot values across the range $[0, 0.99]$ and multiple sample sizes. Three model architectures are evaluated:

• Model 1: Uses 56 engineered summary statistics/features.
• Model 2: Uses convolutional neural networks (CNNs) on 2D heatmap images of the empirical data.
• Model 3: Combines both summary statistics and CNN-extracted features via feature fusion.

Model 3 consistently outperforms both Model 1 and Model 2, as well as the FNN and MINE estimates, achieving significantly reduced bias and variance across the dependency spectrum and under both copula regimes. Critically, the proposed estimator does not require retraining on each new target dataset, in contrast to adaptive neural estimators such as MINE.

The validation protocol ensures that nontrivial dependencies (including low and very high Linfoot values), and varying sample sizes ($n=100,200,500,1000$) are represented, supporting robust generalization within the copula classes considered. Regularization strategies (e.g., dropout, weight pruning) are applied to mitigate overfitting and ensure stable predictions.

Real Data Application: Genetic Dependence in Twin Studies

The pretrained estimator is applied to body mass index (BMI) data from monozygotic (MZ) and dizygotic (DZ) twin pairs, representing a classic scenario where linear correlation vastly underrepresents dependence due to non-Gaussianity and nonlinear effects. The empirical scatterplots for both MZ and DZ groups show clear deviations from Gaussianity, emphasizing the inadequacy of Pearson correlation for these data.

Figure 3: Scatterplot of BMI for monozygotic (MZ) and dizygotic (DZ) twin pairs.

Using the pretrained DNN estimator, the Linfoot informational correlation is quantified as 0.74 for MZ twins and 0.50 for DZ twins, both exceeding the corresponding Pearson values (0.68 for MZ, 0.37 for DZ). Bootstrapped confidence intervals further confirm statistical significance of these differences, providing robust support for a genetic component in BMI determination that is more faithfully captured by the Linfoot correlation. The method also shows moderate sensitivity to monotonic transformations—finite-sample estimates are not strictly invariant, but the effect size is modest.

Discussion and Broader Implications

This paper demonstrates that large pretrained DNNs—fusing statistical summaries and empirical distribution images—consistently outperform classical nonparametric estimators and adaptive neural MI estimators with respect to bias, variance, stability, and computational efficiency. The pretrained architecture eliminates variability due to re-training/resampling, enabling deterministic, reproducible inference once deployed.

However, scope and limitations must be acknowledged. The estimator is validated only for bivariate data generated via Gaussian or Clayton copulas with Gaussian marginals. Extrapolation to other copula families or non-matching marginals likely requires retraining or transfer learning, contingent on the availability of analytic ground-truth Linfoot values. Negative dependence and non-Archimedean dependence structures are not covered in the current regime. Nonetheless, the theoretical invariance properties of the Linfoot measure suggest that with carefully designed simulation protocols (e.g., via homeomorphic transformations), extended model families with known ground truth might be constructed.

From a practical standpoint, this approach provides a turnkey tool for rapid, reliable quantification of dependence in real-world data, enhancing analysis in fields where linear assumptions are rarely tenable. The codebase is made available, facilitating adoption in statistical genetics, neuroscience, biomedicine, and other domains with complex dependency structures.

Conclusion

By leveraging supervised deep learning with copula-calibrated ground truth, this work establishes a significant advance in the nonparametric estimation of the Linfoot informational correlation. The fusion of statistical and image-based features in DNNs yields estimators that are both computationally efficient and statistically robust, realizing low-bias, low-variance inference for a broad subclass of dependencies. The methodological framework is extensible, subject to future theoretical work on ground-truth generation for richer families of bivariate and higher-dimensional dependencies.