An Analysis of Privacy-Aware Personalization Signals by Using Online Evaluation Methods

Abstract: Personalization despite being an effective solution to the problem information overload remains tricky on account of multiple dimensions to consider. Furthermore, the challenge of avoiding overdoing personalization involves estimation of a user's preferences in relation to different queries. This work is an attempt to make inferences about when personalization would be beneficial by relating observable user behavior to his/her social network usage patterns and user-generated content. User behavior on a search system is observed by means of team-draft interleaving whereby results from two retrieval functions are presented in an interleaved manner, and user clicks are utilised to infer preference for a certain retrieval function. This improves upon earlier work which had limited usefulness due to reliance on user survey results; our findings may aid real-time personalization in search systems by detecting a user-related and query-related personalization signals.