Papers
Topics
Authors
Recent
Search
2000 character limit reached

Weighted-Sum of Gaussian Process Latent Variable Models

Published 14 Feb 2024 in stat.ML and cs.LG | (2402.09122v4)

Abstract: This work develops a Bayesian non-parametric approach to signal separation where the signals may vary according to latent variables. Our key contribution is to augment Gaussian Process Latent Variable Models (GPLVMs) for the case where each data point comprises the weighted sum of a known number of pure component signals, observed across several input locations. Our framework allows arbitrary non-linear variations in the signals while being able to incorporate useful priors for the linear weights, such as summing-to-one. Our contributions are particularly relevant to spectroscopy, where changing conditions may cause the underlying pure component signals to vary from sample to sample. To demonstrate the applicability to both spectroscopy and other domains, we consider several applications: a near-infrared spectroscopy dataset with varying temperatures, a simulated dataset for identifying flow configuration through a pipe, and a dataset for determining the type of rock from its reflectance.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (48)
  1. GrandPrix: Scaling up the Bayesian GPLVM for single-cell data. Bioinformatics, 35(1):47–54, 2019.
  2. Ensembles of Gaussian process latent variable models. In 2022 30th European Signal Processing Conference (EUSIPCO), pp.  1467–1471. IEEE, 2022.
  3. Indirect spectral hard modeling for the analysis of reactive and interacting mixtures. Applied spectroscopy, 58(8):975–985, 2004.
  4. Nonlinear spectral unmixing of hyperspectral images using Gaussian processes. IEEE Transactions on Signal Processing, 61(10):2442–2453, 2013. doi: 10.1109/TSP.2013.2245127.
  5. The ASTER spectral library version 2.0. Remote sensing of environment, 113(4):711–715, 2009.
  6. Partial least squares for discrimination. Journal of Chemometrics: A Journal of the Chemometrics Society, 17(3):166–173, 2003.
  7. Standard normal variate transformation and de-trending of near-infrared diffuse reflectance spectra. Applied spectroscopy, 43(5):772–777, 1989.
  8. Analysis of multiphase flows using dual-energy gamma densitometry and neural networks. Nuclear Instruments and Methods in Physics Research Section A: Accelerators, Spectrometers, Detectors and Associated Equipment, 327(2-3):580–593, 1993.
  9. Deep generative endmember modeling: An application to unsupervised spectral unmixing. IEEE Transactions on Computational Imaging, 6:374–384, 2019.
  10. Spectral variability in hyperspectral data unmixing: A comprehensive review. IEEE geoscience and remote sensing magazine, 9(4):223–270, 2021.
  11. Visual scene graphs for audio source separation. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp.  1204–1213, 2021.
  12. Efficient modeling of latent information in supervised learning using Gaussian processes. Advances in Neural Information Processing Systems, 30, 2017.
  13. Variational inference for latent variables and uncertain inputs in Gaussian processes. Journal of Machine Learning Research, 17(42):1–62, 2016.
  14. The UCR time series archive. IEEE/CAA Journal of Automatica Sinica, 6(6):1293–1305, 2019.
  15. Learning GPLVM with arbitrary kernels using the unscented transformation. In International Conference on Artificial Intelligence and Statistics, pp.  451–459. PMLR, 2021.
  16. Incorporating domain knowledge about xrf spectra into neural networks. In Workshop on Perception as Generative Reasoning, NeurIPS, 2019.
  17. Gpytorch: Blackbox matrix-matrix gaussian process inference with gpu acceleration. Advances in neural information processing systems, 31, 2018.
  18. Simple and effective way for data preprocessing selection based on design of experiments. Analytical chemistry, 87(24):12096–12103, 2015.
  19. Self-supervised robust scene flow estimation via the alignment of probability density functions. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pp.  861–869, 2022.
  20. Endmember-guided unmixing network (EGU-Net): A general deep learning framework for self-supervised hyperspectral unmixing. IEEE Transactions on Neural Networks and Learning Systems, 33(11):6518–6531, 2021.
  21. Source separation with deep generative priors. In International Conference on Machine Learning, pp.  4724–4735. PMLR, 2020.
  22. Isometric Gaussian process latent variable model for dissimilarity data. In International Conference on Machine Learning, pp.  5127–5136. PMLR, 2021.
  23. An optimization approach coupling preprocessing with model regression for enhanced chemometrics. Industrial & Engineering Chemistry Research, 62(15):6196–6213, 2023.
  24. Gaussian process latent variable alignment learning. In The 22nd International Conference on Artificial Intelligence and Statistics, pp.  748–757. PMLR, 2019.
  25. Fully automated indirect hard modeling of mixture spectra. Chemometrics and Intelligent Laboratory Systems, 91(2):181–193, 2008.
  26. Generalised gplvm with stochastic variational inference. In International Conference on Artificial Intelligence and Statistics, pp.  7841–7864. PMLR, 2022.
  27. Probabilistic non-linear principal component analysis with Gaussian process latent variable models. Journal of machine learning research, 6(11), 2005.
  28. Refined convergence rates for maximum likelihood estimation under finite mixture models. In International Conference on Machine Learning, pp.  14979–15006. PMLR, 2022.
  29. Supervised extended iterative optimization technology for estimation of powder compositions in pharmaceutical applications: method and lifecycle management. Industrial & Engineering Chemistry Research, 59(21):10072–10081, 2020.
  30. Murphy, K. P. Probabilistic Machine Learning: An introduction, chapter 3.5. MIT Press, 2022. URL probml.ai.
  31. Unsupervised blind source separation with variational auto-encoders. In 2021 29th European Signal Processing Conference (EUSIPCO), pp.  311–315. IEEE, 2021.
  32. Probabilistic predictions for partial least squares using bootstrap. AIChE Journal, pp.  e18071, 2023.
  33. Pytorch: An imperative style, high-performance deep learning library. Advances in neural information processing systems, 32, 2019.
  34. Scikit-learn: Machine learning in python. the Journal of machine Learning research, 12:2825–2830, 2011.
  35. Addressing big data time series: Mining trillions of time series subsequences under dynamic time warping. ACM Transactions on Knowledge Discovery from Data (TKDD), 7(3):1–31, 2013.
  36. Latent Gaussian process with composite likelihoods and numerical quadrature. In International Conference on Artificial Intelligence and Statistics, pp.  3718–3726. PMLR, 2021.
  37. SS-DAC: A systematic framework for selecting the best modeling approach and pre-processing for spectroscopic data. Computers & Chemical Engineering, 128:437–449, 2019.
  38. Consistent estimation of identifiable nonparametric mixture models from grouped observations. Advances in Neural Information Processing Systems, 33:11676–11686, 2020.
  39. EEG-based epileptic seizure detection using GPLV model and multi support vector machine. Journal of Information and Optimization Sciences, 41(1):143–161, 2020.
  40. Tauler, R. Multivariate curve resolution applied to second order data. Chemometrics and intelligent laboratory systems, 30(1):133–146, 1995.
  41. Titsias, M. Variational learning of inducing variables in sparse Gaussian processes. In Artificial intelligence and statistics, pp.  567–574. PMLR, 2009.
  42. Bayesian Gaussian process latent variable model. In Proceedings of the thirteenth international conference on artificial intelligence and statistics, pp.  844–851. JMLR Workshop and Conference Proceedings, 2010.
  43. Discriminative Gaussian process latent variable model for classification. In Proceedings of the 24th international conference on Machine learning, pp.  927–934, 2007.
  44. Gaussian processes for machine learning, volume 2. MIT press Cambridge, MA, 2006.
  45. PLS-regression: a basic tool of chemometrics. Chemometrics and intelligent laboratory systems, 58(2):109–130, 2001.
  46. Influence of temperature on vibrational spectra and consequences for the predictive ability of multivariate models. Analytical chemistry, 70(9):1761–1767, 1998.
  47. Contextually supervised source separation with application to energy disaggregation. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 28, 2014.
  48. ESA-Ariel data challenge NeurIPS 2022: Inferring physical properties of exoplanets from next-generation telescopes. arXiv preprint arXiv:2206.14642, 2022.

Summary

No one has generated a summary of this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Sign Up to Summarize

Paper to Video (Beta)

Whiteboard

No one has generated a whiteboard explanation for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Sign Up to Generate

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now

Continue Learning

We haven't generated follow-up questions for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now

Collections

Sign up for free to add this paper to one or more collections.

User Circle Single Streamline Icon: https://streamlinehq.com Sign Up

Tweets

Sign up for free to view the 4 tweets with 44 likes about this paper.

User Circle Single Streamline Icon: https://streamlinehq.com Sign Up for Free