What Does the Bot Say? Opportunities and Risks of Large Language Models in Social Media Bot Detection

Abstract: Social media bot detection has always been an arms race between advancements in machine learning bot detectors and adversarial bot strategies to evade detection. In this work, we bring the arms race to the next level by investigating the opportunities and risks of state-of-the-art LLMs in social bot detection. To investigate the opportunities, we design novel LLM-based bot detectors by proposing a mixture-of-heterogeneous-experts framework to divide and conquer diverse user information modalities. To illuminate the risks, we explore the possibility of LLM-guided manipulation of user textual and structured information to evade detection. Extensive experiments with three LLMs on two datasets demonstrate that instruction tuning on merely 1,000 annotated examples produces specialized LLMs that outperform state-of-the-art baselines by up to 9.1% on both datasets, while LLM-guided manipulation strategies could significantly bring down the performance of existing bot detectors by up to 29.6% and harm the calibration and reliability of bot detection systems.

• The paper introduces a novel LLM-based framework that integrates metadata, text, and network analysis to boost detection accuracy by up to 9.1%.
• It employs heterogeneous in-context learning techniques, including zero-shot and few-shot prompting, to assess user-generated content and network structures.
• The findings also highlight risks as LLMs can be used to craft more evasive bot behaviors, potentially degrading detection performance by as much as 29.6%.

Opportunities and Risks of LLMs in Social Media Bot Detection

The paper entitled "What Does the Bot Say? Opportunities and Risks of LLMs in Social Media Bot Detection" by Shangbin Feng et al. explores the dual role of LLMs in the context of detecting bots on social media platforms. The authors provide a comprehensive analysis of both the potential advantages and the inherent risks associated with leveraging state-of-the-art LLMs for this purpose.

Overview of Methodology

The authors initiate their study by examining the evolving landscape of social media bot detection, which has historically been an arms race between detection algorithms and increasingly sophisticated bot strategies. The novel contribution of this paper lies in the introduction of LLMs as both powerful tools for detecting bots and as instruments that could potentially be used to enhance the evasiveness of bot accounts.

Opportunities: LLM-based Bot Detectors

To harness the capabilities of LLMs for bot detection, the authors propose a mixture-of-heterogeneous-experts framework. This approach divides the analysis into three user information modalities: metadata, text, and user interaction networks.

1. Metadata: LLMs are used to convert numerical and categorical metadata features into natural language sequences, allowing in-context learning for bot detection.
2. Text: By retrieving similar posts from an annotated dataset, LLMs are instructed with in-context examples to evaluate user-generated content.
3. Network Structure: Utilizing LLMs' graph reasoning capabilities, user following information is integrated into the detection process through structured prompts.

Finally, the modality-specific results from these LLMs are ensembled to arrive at a consensus through majority voting.

Risks: Potential for LLM-guided Bot Designs

The paper also explores the risks by demonstrating how LLMs can be used to make social bots more sophisticated. This is achieved through:

• Textual Manipulation: LLMs generate alternate versions of user posts to appear more human-like, employing methods such as zero-shot prompting, few-shot rewriting, and interactive refinement with external classifiers.
• Structural Manipulation: By suggesting which users to follow or unfollow, LLMs can subtly alter the perceived legitimacy of bot accounts in the social network.

Experimental Results

The experiments conducted with three LLMs—Mistral-7B, LLaMA2-70B, and ChatGPT—on two established datasets, Twibot-20 and Twibot-22, illustrate the dual potential of LLMs. Bot detection systems enhanced with LLMs reporting an improvement of up to 9.1% in accuracy over state-of-the-art baselines. This superior performance underscores the efficacy of instruction-tuning LLMs with a limited set of annotated examples.

Conversely, LLM-aided bot manipulation strategies were shown to degrade the performance of existing detectors by as much as 29.6%, highlighting the real adversarial threat posed by advanced LLMs in bot design.

Implications and Future Directions

This research highlights a critical juncture in the field of social media bot detection. The successful application of LLMs presents opportunities for more accurate and resilient detection systems. However, the ease with which LLMs can facilitate adversarial strategies poses a significant ethical challenge. Addressing this dual-use nature may require advanced adversarial training techniques and continuous updating of detection models to mitigate risks.

In conclusion, future research could focus on exploring interdisciplinary approaches incorporating insights from domains such as cybersecurity and network science to develop holistic defense mechanisms against sophisticated bot attacks while ensuring that these systems remain transparent and ethically responsible.