AI-Powered Augmented Reality as a Threat Vector for Human Manipulation

Abstract: Augmented Reality (AR) is a powerful perceptual technology that can alter what users see, hear, feel, and experience throughout their daily lives. When combined with the speed and flexibility of context-aware generative AI, the power is greatly expanded, allowing individual users to be targeted with custom-generated AR experiences that are instantly tailored to who they are, where they are, and what they are doing. This can transform the physical world into a magical place, but only if the augmentation of a user's environment is enacted for their personal benefit and best interests. Instead, if AI-powered AR systems are controlled by unregulated third parties, such as large corporations or state actors, individually adaptive AR experiences could be deployed as a dangerous form of targeted influence. In fact, if the industry adopts an advertising business model for AI-powered AR devices, context-aware generative influence could become a widely used manipulative path for promotion of products and services in the physical world. Worse, similar techniques could be used for political influence, propaganda, and disinformation. This chapter reviews the power and flexibility of AI-generated augmented reality, explores the risks that emerge when used for persuasion, manipulation, or influence, and proposes policy directions to mitigate these risks.