Walkthrough of Anthropomorphic Features in AI Assistant Tools

Abstract: In this paper, we attempt to understand the anthropomorphic features of chatbot outputs and how these features provide a discursive frame for human-AI interactions. To do so, we explore the use of a prompt-based walkthrough method with two phases: (1) interview-style prompting to reveal the chatbots' context of expected use and (2) roleplaying-type prompting to evoke everyday use scenarios and typical chatbot outputs. We applied this method to catalogue anthropomorphic features across four different LLM chatbots, finding that anthropomorphism was exhibited as both subjective language and a sympathetic conversational tone. We also found that socio-emotional cues in prompts increase the incidence of anthropomorphic expressions in outputs. We argue that the prompt-based walkthrough method was successful in stimulating social role performance in LLM chatbots and in eliciting a variety of anthropomorphic features, making it useful in the study of interaction-based algorithmic harms where users project inappropriate social roles onto LLM-based tools.