2000 character limit reached
Contrastive independent component analysis
Published 2 Jul 2024 in math.ST, math.AG, stat.ML, and stat.TH | (2407.02357v2)
Abstract: In recent years, there has been growing interest in jointly analyzing a foreground dataset, representing an experimental group, and a background dataset, representing a control group. The goal of such contrastive investigations is to identify salient features in the experimental group relative to the control. Independent component analysis (ICA) is a powerful tool for learning independent patterns in a dataset. We generalize it to contrastive ICA (cICA). For this purpose, we devise a new linear algebra based tensor decomposition algorithm, which is more expressive but just as efficient and identifiable as other linear algebra based algorithms. We establish the identifiability of cICA and demonstrate its performance in finding patterns and visualizing data, using synthetic, semi-synthetic, and real-world datasets, comparing the approach to existing methods.
- Contrastive variational autoencoder enhances salient features. arXiv preprint arXiv:1902.04601, 2019.
- Contrastive principal component analysis. arXiv preprint arXiv:1709.06716, 2017.
- Exploring patterns enriched in a dataset with contrastive principal component analysis. Nature communications, 9(1):2134, 2018.
- Handbook of Blind Source Separation: Independent component analysis and applications. Academic press, 2010.
- Pierre Comon. Independent component analysis, a new concept? Signal processing, 36(3):287–314, 1994.
- On generic identifiability of symmetric tensors of subgeneric rank. Transactions of the American Mathematical Society, 369(6):4021–4042, 2017.
- Blind beamforming for non-Gaussian signals. In IEE proceedings F (radar and signal processing), volume 140, pages 362–370. IET, 1993.
- Fourth-order cumulant-based blind identification of underdetermined mixtures. IEEE Transactions on Signal Processing, 55:2965–2973, 2007.
- Independent component analysis and (simultaneous) third-order tensor diagonalization. IEEE Transactions on Signal Processing, 49(10):2262–2271, 2001.
- Krzysztof Domino. The use of fourth order cumulant tensors to detect outlier features modelled by a t-student copula. arXiv preprint arXiv:1804.00541, 2018.
- J. Eriksson and V. Koivunen. Identifiability, separability, and uniqueness of linear ICA models. IEEE Signal Processing Letters, 11(7):601–604, 2004.
- NPSA: Nonorthogonal principal skewness analysis. IEEE Transactions on Image Processing, 29:6396–6408, 2020.
- Wolfgang Hackbusch. Tensor spaces and numerical tensor calculus, volume 42. Springer, 2012.
- Self-organizing feature maps identify proteins critical to learning in a mouse model of Down syndrome. PloS one, 10(6):e0129126, 2015.
- Unsupervised feature extraction by time-contrastive learning and nonlinear ICA. Advances in neural information processing systems, 29, 2016.
- Nonlinear ICA using auxiliary variables and generalized contrastive learning. In The 22nd International Conference on Artificial Intelligence and Statistics, pages 859–868. PMLR, 2019.
- A. Hirschowitz J. Alexander. Polynomial interpolation in several variables. J. Algebraic Geom. 4(4) (1995), 1995.
- Subspace power method for symmetric tensor decomposition and generalized PCA. arXiv preprint arXiv:1912.04007, 2019.
- Qi Lyu and Xiao Fu. On finite-sample identifiability of contrastive learning-based nonlinear independent component analysis. In International Conference on Machine Learning, pages 14582–14600. PMLR, 2022.
- Toward the identifiability of comparative deep generative models. arXiv preprint arXiv:2401.15903, 2024.
- Probabilistic contrastive principal component analysis. arXiv preprint arXiv:2012.07977, 2020.
- Cumulant component analysis: a simultaneous generalization of PCA and ICA. CASTA2008, 18, 2008.
- Peter McCullagh. Tensor methods in statistics: Monographs on statistics and applied probability. Chapman and Hall/CRC, 2018.
- Elina Robeva. Orthogonal decomposition of symmetric tensors. SIAM Journal on Matrix Analysis and Applications, 37(1):86–102, 2016.
- Peter J Rousseeuw. Silhouettes: a graphical aid to the interpretation and validation of cluster analysis. Journal of computational and applied mathematics, 20:53–65, 1987.
- Comparative single-cell transcriptomic analysis of primate brains highlights human-specific regulatory evolution. Nature Ecology & Evolution, 7(11):1930–1943, 2023.
- Unsupervised learning with contrastive latent variable models. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 33, pages 4862–4869, 2019.
- Sequential unfolding SVD for tensors with applications in array signal processing. IEEE Transactions on Signal Processing, 57(12):4719–4733, 2009.
- Linear causal disentanglement via interventions. In International Conference on Machine Learning, pages 32540–32560. PMLR, 2023.
- Unpaired multi-domain causal representation learning. Advances in Neural Information Processing Systems, 36, 2024.
- Moment matching deep contrastive latent variable models. arXiv preprint arXiv:2202.10560, 2022.
- Hermann Weyl. Das asymptotische verteilungsgesetz der eigenwerte linearer partieller differentialgleichungen (mit einer anwendung auf die theorie der hohlraumstrahlung). Mathematische Annalen, 71(4):441–479, 1912.
- Identifiability of overcomplete independent component analysis. arXiv preprint arXiv:2401.14709, 2024.
- CausalVAE: Disentangled representation learning via neural structural causal models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 9593–9602, 2021.
- Contrastive learning using spectral methods. Advances in Neural Information Processing Systems, 26, 2013.
- Massively parallel digital transcriptional profiling of single cells. Nature communications, 8(1):14049, 2017.
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
Top Community Prompts
Collections
Sign up for free to add this paper to one or more collections.