Toward a Team of AI-made Scientists for Scientific Discovery from Gene Expression Data

Abstract: Machine learning has emerged as a powerful tool for scientific discovery, enabling researchers to extract meaningful insights from complex datasets. For instance, it has facilitated the identification of disease-predictive genes from gene expression data, significantly advancing healthcare. However, the traditional process for analyzing such datasets demands substantial human effort and expertise for the data selection, processing, and analysis. To address this challenge, we introduce a novel framework, a Team of AI-made Scientists (TAIS), designed to streamline the scientific discovery pipeline. TAIS comprises simulated roles, including a project manager, data engineer, and domain expert, each represented by a LLM. These roles collaborate to replicate the tasks typically performed by data scientists, with a specific focus on identifying disease-predictive genes. Furthermore, we have curated a benchmark dataset to assess TAIS's effectiveness in gene identification, demonstrating our system's potential to significantly enhance the efficiency and scope of scientific exploration. Our findings represent a solid step towards automating scientific discovery through LLMs.

• The paper introduces TAIS, a multi-agent system using LLMs to automate gene expression analysis and predict disease-associated genes.
• It employs Lasso regression with eigenvalue gap analysis to isolate predictive genes while accounting for confounding factors such as age and gender.
• Benchmarking on the GenQEX dataset demonstrates TAIS's competitive precision and recall, highlighting potential for enhancing personalized treatment strategies.

Toward a Team of AI-made Scientists for Scientific Discovery from Gene Expression Data

Introduction

In the pursuit of advancing scientific discovery through automation, gene expression datasets have been identified as crucial for understanding disease-predictive genes, especially for complex diseases such as cancer. Traditional methods of gene analysis require significant human expert intervention in dataset selection, processing, and analysis. This paper proposes a framework, a Team of AI-made Scientists (TAIS), to automate and streamline these processes using a multi-agent system composed of various roles modeled by LLMs. These roles mimic human scientists working collaboratively to predict disease-associated genes more efficiently.

System Architecture

TAIS is structured to simulate the collaborative dynamics of a human research team comprising five key roles: Project Manager, Data Engineer, Domain Expert, Statistician, and Code Reviewer. Each role is fulfilled by an LLM acting as an agent responsible for specific tasks.

The Project Manager initiates the setup, decomposing the overarching problem into manageable subtasks aligned with each role's capabilities. For example, gene data preprocessing tasks are assigned to the Data Engineer, who collaborates with the Domain Expert for technical guidance. Meanwhile, the Statistician, supported by the Code Reviewer, carries out regression analyses on processed datasets.

Figure 1: The overview of the Team of the AI-made Scientists (TAIS). The illustration starts from the top right corner where the user uses the system. The question goes to the project manager. The project manager further decomposes the tasks and assigns tasks to different AI-made scientists, illustrated in the yellow area. The blue area shows the details of how the statistician analyzes the data.

Collaboration and Task Execution

The collaboration among agents is pivotal for ensuring task precision. The Data Engineer, while performing data preprocessing, frequently interfaces with the Domain Expert to obtain contextual understanding crucial for tasks involving complex datasets from public repositories like GEO and TCGA. This interaction ensures the accurate handling of biomedical terminologies and experimental setups.

Figure 2: The collaboration between Data Engineer and Code Reviewer.

Additionally, the program-and-review dynamic between the Data Engineer, Statistician, and Code Reviewer is essential. Code Reviewer evaluates all produced code for adherence to standards and scientific rigour, providing iterative feedback to resolve any issues before proceeding.

Regression Analysis for Gene Identification

The regression model used within TAIS adapts Lasso regression for variable selection from high-dimensional gene expression data. This model efficiently isolates predictive genes by suppressing less informative features. Confounding factor detection is done via eigenvalue gap analysis of the covariance matrix, guiding decisions on correction processes during regression.

The incorporation of external conditions such as age and gender is modeled by extending the basic regression framework to include these variables as additional matrices, allowing TAIS to further refine gene-disease relationships.

Benchmark Creation and Evaluation

To assess TAIS's capability, a benchmark dataset titled Genetic Question Exploration (GenQEX) was developed, including 457 trait-condition pairs derived from a pre-defined list of biomedical entities. These questions directed TAIS to identify predictive genes while considering various potential confounding diseases or demographic factors.

Results and Discussion

TAIS effectively streamlined the discovery of disease-predictive genes, showing competitive precision and recall across various complex scenarios. While the system outperformed traditional methods requiring heavy manual input, particular challenges remain in fully realizing the potential of multi-step and two-step regression analyses.

The case study on Pancreatic Cancer with Vitamin D Levels demonstrated TAIS's proficiency in identifying genes validated by existing biomedical literature. This achievement emphasizes the potential of TAIS to enhance personalized treatment strategies by providing insights into how conditions affect gene-disease relationships.

Conclusion

This paper presents a novel framework through TAIS, using LLM agents to automate labor-intensive elements of scientific research. The implementation highlights steps towards reducing human dependency in data analysis workflows. While promising, continued improvement and expansion of capabilities are necessary to address limitations inherent in fully automated scientific discovery systems. The benchmark provides a solid foundation for future methods to compare and enhance upon TAIS's existing framework.