Fake Friends and Sponsored Ads: The Risks of Advertising in Conversational Search

Abstract: Digital commerce thrives on advertising, with many of the largest technology companies relying on it as a significant source of revenue. However, in the context of information-seeking behavior, such as search, advertising may degrade the user experience by lowering search quality, misusing user data for inappropriate personalization, potentially misleading individuals, or even leading them toward harm. These challenges remain significant as conversational search technologies, such as ChatGPT, become widespread. This paper critically examines the future of advertising in conversational search, utilizing several speculative examples to illustrate the potential risks posed to users who seek guidance on sensitive topics. Additionally, it provides an overview of the forms that advertising might take in this space and introduces the "fake friend dilemma," the idea that a conversational agent may exploit unaligned user trust to achieve other objectives. This study presents a provocative discussion on the future of online advertising in the space of conversational search and ends with a call to action.