rLLM: Relational Table Learning with LLMs

Abstract: We introduce rLLM (relationLLM), a PyTorch library designed for Relational Table Learning (RTL) with LLMs. The core idea is to decompose state-of-the-art Graph Neural Networks, LLMs, and Table Neural Networks into standardized modules, to enable the fast construction of novel RTL-type models in a simple "combine, align, and co-train" manner. To illustrate the usage of rLLM, we introduce a simple RTL method named \textbf{BRIDGE}. Additionally, we present three novel relational tabular datasets (TML1M, TLF2K, and TACM12K) by enhancing classic datasets. We hope rLLM can serve as a useful and easy-to-use development framework for RTL-related tasks. Our code is available at: https://github.com/rllm-project/rllm.

• The paper introduces rLLM, a modular framework that standardizes GNNs, LLMs, and TNNs for efficient relational table learning.
• It demonstrates the BRIDGE algorithm, which outperforms traditional single-table models by leveraging inter-table relationships.
• New datasets like TML1M, TLF2K, and TACM12K provide robust benchmarks for evaluating advanced RTL methods.

rLLM: Relational Table Learning with LLMs

The paper "rLLM: Relational Table Learning with LLMs" presents rLLM, a PyTorch-based library designed to facilitate Relational Table Learning (RTL) using LLMs. This system deconstructs Graph Neural Networks (GNNs), LLMs, and Table Neural Networks (TNNs) into standardized modules, enabling the rapid development of novel RTL models through a flexible "combine, align, and co-train" approach.

Overview

The main contributions of the paper include:

1. rLLM System: A comprehensive framework to integrate and leverage GNNs, LLMs, and TNNs for RTL tasks.
2. BRIDGE Algorithm: An example RTL method demonstrating the practical usage of the rLLM framework.
3. New Datasets: Introduction of three novel relational tabular datasets (TML1M, TLF2K, and TACM12K) for RTL.

These contributions collectively aim to streamline the development process of RTL methods and provide valuable resources for the research community.

System Architecture

The rLLM framework is composed of three primary layers:

1. Data Engine Layer: Focuses on defining fundamental data structures and processing workflows for relational table data, handling both table and graph data in a decoupled and flexible manner.
2. Module Layer: Decomposes operations of GNNs, LLMs, and TNNs into standard submodules, facilitating diverse and complex data processing tasks.
3. Model Layer: Offers strategies to develop RTL models through combining, aligning, and co-training various modules, allowing for robust and adaptable model construction.

BRIDGE Algorithm

The BRIDGE (Basic Relational table-Data LearninG FramEwork) algorithm exemplifies the application of the rLLM framework. It integrates TNNs for processing table data and GNNs for modeling relationships between tables defined by foreign keys. This dual approach enables BRIDGE to leverage multiple tables and their interrelationships effectively, resulting in superior performance in RTL tasks.

Novel Datasets

The introduction of the SJTUTables collection includes three datasets enhanced from classic datasets:

• TML1M: Derived from MovieLens 1M, this dataset includes enriched movie data and an age range classification task.
• TLF2K: Based on LastFM 2K, it features comprehensive artist metadata and a genre classification task.
• TACM12K: Enhanced ACM dataset with detailed paper and author information, designed for conference classification tasks.

These datasets offer well-organized and balanced splits, providing a robust foundation for designing and evaluating RTL methods.

Experimental Evaluation

The paper provides experimental results on the TML1M dataset, comparing the BRIDGE algorithm with traditional single-tabular TNNs such as TabTransformer, TabNet, and FT-Transformer. BRIDGE demonstrated superior performance, highlighting the system's ability to harness the interconnected nature of relational tables effectively.

Methods	TML1M	TLF2K	TACM12K
Random	0.144±0.01	0.091±0.03	0.075±0.0
TabTransformer	0.347±0.02	0.1370±0.08	0.091±0.01
TabNet	0.259±0.08	0.1346±0.03	0.135±0.01
FT-Transformer	0.352±0.02	0.1319±0.01	0.099±0.01
BRIDGE	0.362±0.03	0.422±0.03	0.256±0.01

The results indicate that traditional methods were limited to learning from individual tables, whereas BRIDGE effectively utilized additional relational information, resulting in a performance boost across the datasets.

Implications and Future Directions

The rLLM framework and the BRIDGE algorithm underscore the potential of combining GNNs, LLMs, and TNNs for RTL tasks. Practically, this system can significantly simplify the process of developing sophisticated RTL models, catering to diverse applications where relational data is prevalent. The theoretically modular nature of rLLM allows for flexibility and scalability, fostering innovation in RTL research.

Potential future directions include optimizing the data structures for enhanced efficiency and integrating more advanced methods. Collaboration with the research community could further expand the applicability and robustness of the rLLM framework.

Conclusion

The rLLM framework represents a structured approach to leverage the synergy between GNNs, LLMs, and TNNs for Relational Table Learning. The BRIDGE algorithm and the introduction of new datasets highlight the framework's practical utility and the potential to pave the way for future advancements in this domain. Researchers and engineers are encouraged to collaborate and contribute to extending the rLLM framework, thereby accelerating the progression of RTL methodologies.