LLM-powered Multi-agent Framework for Goal-oriented Learning in Intelligent Tutoring System

Abstract: Intelligent Tutoring Systems (ITSs) have revolutionized education by offering personalized learning experiences. However, as goal-oriented learning, which emphasizes efficiently achieving specific objectives, becomes increasingly important in professional contexts, existing ITSs often struggle to deliver this type of targeted learning experience. In this paper, we propose GenMentor, an LLM-powered multi-agent framework designed to deliver goal-oriented, personalized learning within ITS. GenMentor begins by accurately mapping learners' goals to required skills using a fine-tuned LLM trained on a custom goal-to-skill dataset. After identifying the skill gap, it schedules an efficient learning path using an evolving optimization approach, driven by a comprehensive and dynamic profile of learners' multifaceted status. Additionally, GenMentor tailors learning content with an exploration-drafting-integration mechanism to align with individual learner needs. Extensive automated and human evaluations demonstrate GenMentor's effectiveness in learning guidance and content quality. Furthermore, we have deployed it in practice and also implemented it as an application. Practical human study with professional learners further highlights its effectiveness in goal alignment and resource targeting, leading to enhanced personalization. Supplementary resources are available at https://github.com/GeminiLight/gen-mentor.

• The paper presents GenMentor, a novel multi-agent framework that leverages LLMs to align learning goals with requisite skills.
• It employs Chain-of-Thought reasoning and dynamic learner modeling to identify skill gaps and continuously adapt to real-time feedback.
• Automated and human evaluations demonstrate enhanced content personalization, learner engagement, and overall goal-oriented learning efficacy.

LLM-powered Multi-agent Framework for Goal-oriented Learning in ITS

Introduction

The integration of LLMs into Intelligent Tutoring Systems (ITSs) paves the way for highly personalized educational experiences. The paper "LLM-powered Multi-agent Framework for Goal-oriented Learning in Intelligent Tutoring System" (2501.15749) introduces GenMentor, a novel multi-agent framework that utilizes LLMs to enhance goal-oriented learning within ITS. Unlike traditional ITSs, which often face challenges in adaptability and goal alignment, GenMentor leverages LLMs to overcome these limitations by providing a tailored educational experience that aligns with specific learning objectives.

Figure 1: Comparison of three types of ITS Paradigms.

The GenMentor Framework

GenMentor distinguishes itself through its multi-agent framework, which distributes tasks across specialized LLM agents. This architecture is designed to map learning goals to requisite skills, model learner profiles dynamically, and curate personalized learning paths. These agents work collaboratively to fine-tune the educational content delivered to learners, addressing traditional ITS limitations such as static curricula and fragmented data management.

Figure 2: Overview of the GenMentor: An LLM-powered multi-agent framework for goal-oriented learning in ITS.

Skill Gap Identification

A critical component of GenMentor is its ability to accurately identify skill gaps using a fine-tuned LLM model. This model is trained on a bespoke goal-to-skill dataset, enabling it to map high-level learning goals to the specific skills learners need to acquire. This mapping process is enhanced through Chain-of-Thought (CoT) reasoning, which enriches the model's understanding of abstract goals by breaking them down into explicit tasks and skills.

Dynamic Learner Modeling

GenMentor integrates a dynamic learner modeling system capable of adapting to real-time feedback. This system continuously refines learner profiles by analyzing various factors, including cognitive status, learning preferences, and behavioral patterns. This adaptive profiling ensures that the learning experience evolves with the learner's progress, leading to more effective personalization.

Figure 3: An illustration of dynamic learner modeling.

Personalized Resource Delivery

The system's ability to deliver personalized resources is enhanced by the collaborative interaction between its LLM agents and the adaptive learner model. GenMentor's path scheduler and content creator ensure that the learning materials are both comprehensive and curated to match the learner's evolving profile. The content creation process involves exploration, drafting, and integration phases, designed to align educational content closely with the learner's goals and preferences.

Evaluation and Practical Deployment

GenMentor's effectiveness was validated through automated evaluations using GPT-4o and Llama LLMs, alongside human evaluations. Results from these evaluations showed significant improvements in skill gap identification, learning path engagement, and content personalization compared to traditional methods. Additionally, deployed in practical settings, GenMentor facilitated efficient and goal-oriented learning, further underscoring its practical utility.

Figure 4: Questionnaire results from 20 participants (questions shortened for clarity). Gray texts are means and std. deviations.

Conclusion

GenMentor represents a significant advancement in the domain of Intelligent Tutoring Systems. By combining the power of LLMs with a multi-agent framework, it effectively addresses the need for goal-oriented learning experiences. Its ability to dynamically model learner profiles and personalize learning resources underscores the transformative potential of LLMs in educational contexts. Future developments could explore further enhancements in real-time adaptability and broader applications in various professional learning environments.