Large Language Models for User Interest Journeys

Abstract: LLMs have shown impressive capabilities in natural language understanding and generation. Their potential for deeper user understanding and improved personalized user experience on recommendation platforms is, however, largely untapped. This paper aims to address this gap. Recommender systems today capture users' interests through encoding their historical activities on the platforms. The generated user representations are hard to examine or interpret. On the other hand, if we were to ask people about interests they pursue in their life, they might talk about their hobbies, like I just started learning the ukulele, or their relaxation routines, e.g., I like to watch Saturday Night Live, or I want to plant a vertical garden. We argue, and demonstrate through extensive experiments, that LLMs as foundation models can reason through user activities, and describe their interests in nuanced and interesting ways, similar to how a human would. We define interest journeys as the persistent and overarching user interests, in other words, the non-transient ones. These are the interests that we believe will benefit most from the nuanced and personalized descriptions. We introduce a framework in which we first perform personalized extraction of interest journeys, and then summarize the extracted journeys via LLMs, using techniques like few-shot prompting, prompt-tuning and fine-tuning. Together, our results in prompting LLMs to name extracted user journeys in a large-scale industrial platform demonstrate great potential of these models in providing deeper, more interpretable, and controllable user understanding. We believe LLM powered user understanding can be a stepping stone to entirely new user experiences on recommendation platforms that are journey-aware, assistive, and enabling frictionless conversation down the line.

• The paper demonstrates the use of personalized clustering (ICPC) and prompt-tuning to effectively capture and label user interest journeys.
• It leverages LLMs like LaMDa and PaLM, outperforming traditional methods by generating nuanced, human-like journey descriptions.
• The findings reveal that larger models and curated datasets significantly improve personalization and offer deeper insights in recommendation systems.

LLMs Enhancing User Interest Journeys in Recommendation Systems

Overview

In the burgeoning field of recommendation systems, the utilization of LLMs holds significant potential to advance personalization and user experience. Google Inc.’s recent study explores this potential, focusing on how LLMs can be effectively deployed to understand and articulate user interest journeys. Unlike traditional models which often rely on interpreting historical user activity data, this approach hinges on reasoning through user interactions on recommendation platforms to deduce nuanced and personalized user interest journeys. Through extensive experiments, the research showcases the capability of LLMs to describe user interests in a manner reminiscent of human conversation, thereby fostering a deeper level of personalized user experience.

Methodology

The foundation of this research rests on two main components: personalized extraction of user interest journeys and their subsequent naming through LLMs. The process commences with the clustering of user interactions to identify coherent interest journeys, following which techniques such as few-shot prompting, prompt-tuning, and fine-tuning are employed to refine LLMs' capacity to generate descriptive, human-like names for these journeys.

Designing the Journey Service

The journey service crafted for this study encapsulates the essence of personalization in recommendation systems. It delineates user journeys by aggregating user-item interactions into clusters, which are then named through the adept application of LLMs. The study introduces an algorithm, Infinite Concept Personalized Clustering (ICPC), optimized to perform personalized clustering by leveraging salient terms within user interactions, thereby achieving thematic coherence.

Leveraging LLMs

Building on LLMs such as LaMDa and PaLM, the study explores various methodologies to tune these models for the domain of user interest journeys. The exploration spans few-shot prompting, prompt-tuning, and fine-tuning techniques, each examined for its effectiveness in generating accurate and nuanced journey names. This exploration is bolstered by the use of high-quality domain-specific datasets for model alignment, notably utilizing user-curated playlists as proxies.

Results and Findings

The study presents several key findings:

• Personalized Clustering Efficiency: The proposed method, ICPC, demonstrates superior ability in clustering user interactions into coherent thematic journeys compared to non-personalized baselines.
• Effectiveness of LLMs in Journey Naming: Through prompt-tuning based on quality datasets, LLMs can generate interpretable and nuanced names for user interest journeys. Furthermore, prompt-tuning outperforms both few-shot prompting and fine-tuning in generalization capability across different experimental setups.
• Model Variation and Data Source Impact: The study underscores the significance of the underlying model architecture and the data used for prompt-tuning in determining the quality of generated journey names. Larger models and expert-curated collection-based prompting data generally yield better results.

Implications and Future Directions

The application of LLMs to understand and articulate user interest journeys presents a promising avenue toward enhancing personalized recommendations. This approach not only offers a deeper understanding of user intents but also enables more meaningful and engaging user experiences on recommendation platforms. Future research may explore the integration of journey-aware recommendations, the scalability of personalized clustering algorithms, and the exploration of user feedback mechanisms for continuous model improvement.

In conclusion, this study provides compelling evidence of the potential benefits of integrating LLMs into recommendation systems to encapsulate user interest journeys accurately. Through sophisticated personalized clustering and the nuanced naming of user journeys, LLMs can indeed pave the way for a new era of personalized user experiences in recommendation systems.