Extent of zero-shot LLM reasoning from textual reviews

Characterize the extent to which zero-shot large language models, when conditioned on textual reviews of users and items, can reason about subtle user preferences and identify salient item attributes that appeal to individual users in review-aware rating prediction.

Background

The paper investigates the role of textual reviews in modern recommender systems that leverage LLMs. While LLMs are known for strong generalization and reasoning abilities, most existing LLM-based recommendation methods overlook review text and rely mainly on interaction histories.

In the Methods section, the authors define a zero-shot prompt that presents user and item reviews and asks the LLM to predict ratings. However, they explicitly note uncertainty regarding how effectively zero-shot LLMs can reason over review text to infer nuanced user preferences and salient item attributes relevant to recommendation.