Factor Analysis with Correlated Topic Model for Multi-Modal Data

Abstract: Integrating various data modalities brings valuable insights into underlying phenomena. Multimodal factor analysis (FA) uncovers shared axes of variation underlying different simple data modalities, where each sample is represented by a vector of features. However, FA is not suited for structured data modalities, such as text or single cell sequencing data, where multiple data points are measured per each sample and exhibit a clustering structure. To overcome this challenge, we introduce FACTM, a novel, multi-view and multi-structure Bayesian model that combines FA with correlated topic modeling and is optimized using variational inference. Additionally, we introduce a method for rotating latent factors to enhance interpretability with respect to binary features. On text and video benchmarks as well as real-world music and COVID-19 datasets, we demonstrate that FACTM outperforms other methods in identifying clusters in structured data, and integrating them with simple modalities via the inference of shared, interpretable factors.

Factor Analysis with Correlated Topic Model for Multi-Modal Data

This paper presents FACTM, a novel approach to factor analysis (FA) tailored for multi-modal and structured data. FACTM integrates FA with Correlated Topic Models (CTM) using a Bayesian framework optimized through variational inference, addressing the limitations of existing methods in handling structured data alongside simpler data modalities. The development and evaluation of FACTM are extensively detailed, reflecting a rigorous approach to bridging the gap between FA and CTM methodologies.

Background and Methodological Innovations

Factor Analysis traditionally operates on datasets described by feature vectors, but often struggles with structured data where sample data points exhibit clustering, such as text or sequencing data. FACTM innovatively extends FA by incorporating CTM, a method well-suited to mining structured data, like text documents, where words cluster into topics. FACTM links cluster prevalences in structured views with simple modalities using sample-specific modification vectors, overcoming the inherent limitations of standard FA models.

Another significant aspect of FACTM is its use of a supervised orientation method to facilitate factor interpretability, addressing FA's identifiability issues like rotation invariance.

Results and Implications

FACTM was benchmarked against existing methods on diverse datasets, ranging from text and video benchmarks to real-world domains like music and COVID-19 patient data. The results show FACTM's superior performance in accurately identifying clusters in structured data and integrating them with simple modalities. This makes FACTM especially adept at handling complex datasets where structured and simple data types coexist, allowing for a synergistic integration of heterogeneous data sources.

The successful application of FACTM to COVID-19 datasets demonstrated its utility in biological contexts, identifying cell clusters within single-cell RNA sequencing data. FACTM elucidated meaningful biological clustering, indicating its potential for advancing research into complex biological phenomena.

Future Directions and Considerations

While FACTM addresses key limitations of existing methods, it assumes linear dependencies between latent factors and data views, which could be extended to nonlinear dynamics for greater model expressiveness. Additionally, the specification of hyperparameters may be further refined, potentially through automatic learning techniques, to optimize model performance without manual tuning.

Overall, FACTM sets a significant precedent for merging FA with structured data analysis methodologies, contributing a flexible tool for the integration of multi-modal datasets. Its implications span practical applications in medical research and theoretical advancements in machine learning methodologies, offering promising avenues for future developments in AI-driven data analysis.