Unsupervised detection of coordinated information operations in the wild
Abstract: This paper introduces and tests an unsupervised method for detecting novel coordinated inauthentic information operations (CIOs) in realistic settings. This method uses Bayesian inference to identify groups of accounts that share similar account-level characteristics and target similar narratives. We solve the inferential problem using amortized variational inference, allowing us to efficiently infer group identities for millions of accounts. We validate this method using a set of five CIOs from three countries discussing four topics on Twitter. Our unsupervised approach increases detection power (area under the precision-recall curve) relative to a naive baseline (by a factor of 76 to 580), relative to the use of simple flags or narratives on their own (by a factor of 1.3 to 4.8), and comes quite close to a supervised benchmark. Our method is robust to observing only a small share of messaging on the topic, having only weak markers of inauthenticity, and to the CIO accounts making up a tiny share of messages and accounts on the topic. Although we evaluate the results on Twitter, the method is general enough to be applied in many social-media settings.
- Linguistic cues to deception: Identifying political trolls on social media. In Proceedings of the international AAAI conference on web and social media, Volume 13 (pp. 15–25).
- Amortized variational inference for simple hierarchical models. Advances in Neural Information Processing Systems, 34, 21388–21399.
- A survey on multimodal disinformation detection. arXiv preprint arXiv:2103.12541.
- Content-based features predict social media influence operations. Science Advances, 6(30), eabb5824.
- Relatio: Text semantics capture political and economic narratives. Political Analysis, 32(1), 115–132.
- Bishop, C. M. (2013). Model-based machine learning. Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences, 371(1984), 20120222.
- Variational inference: A review for statisticians. Journal of the American statistical Association, 112(518), 859–877.
- Bushwick, S. (2022). Russia’s information war is being waged on social media platforms. Scientific American.
- Uncovering large groups of active malicious accounts in online social networks. In Proceedings of the 2014 ACM SIGSAC Conference on Computer and Communications Security (pp. 477–488).
- Narrative persuasion and overcoming resistance. In E. S. Knowles & J. A. Linn (Eds.), Resistance and persuasion (pp. 175–191). Lawrence Erlbaum Associates.
- Donath, J. S. et al. (1999). Identity and deception in the virtual community. Communities in cyberspace, 1996, 29–59.
- Edgitt, S. (2017). Opening remarks from twitter general counsel. In US Senate Committee on the Judiciary, Subcommittee on Crime and Terrorism.
- How disinformation evolved in 2020. Brookings.
- Malreg: Detecting and analyzing malicious retweeter groups. In Proceedings of the ACM India Joint International Conference on Data Science and Management of Data (pp. 61–69).
- Still out there: Modeling and identifying russian troll accounts on twitter. In 12th ACM Conference on Web Science (pp. 1–10).
- Batch normalization: Accelerating deep network training by reducing internal covariate shift. In International conference on machine learning (pp. 448–456).
- Detecting troll behavior via inverse reinforcement learning: A case study of russian trolls in the 2016 us election. In Proceedings of the International AAAI Conference on Web and Social Media, Volume 14 (pp. 417–427).
- Hacked and hoaxed: Tactics of an iran-linked operation to influence black lives matter narratives on twitter. Available at: https://cyber.fsi.stanford.edu/io/news/twitter-takedown-iran-october-2020 (Archived: https://archive.ph/ikDJh).
- Uncovering coordinated networks on social media: methods and case studies. In Proceedings of the international AAAI conference on web and social media, Volume 15 (pp. 455–466).
- CooRTweet: Coordinated Networks Detection on Social Media. R package version 1.3.3.
- Csi: A hybrid deep model for fake news detection. In Proceedings of the 2017 ACM on Conference on Information and Knowledge Management (pp. 797–806).
- Identifying coordinated accounts on social media through hidden influence and group behaviours. arXiv preprint arXiv:2008.11308.
- Covid-19 vaccines: characterizing misinformation campaigns and vaccine hesitancy on twitter. arXiv preprint arXiv:2106.08423.
- Fake news detection on social media: A data mining perspective. ACM SIGKDD explorations newsletter, 19(1), 22–36.
- Automatic detection of influential actors in disinformation networks. Proceedings of the National Academy of Sciences, 118(4).
- Autoencoding variational inference for topic models. arXiv preprint arXiv:1703.01488.
- Dropout: a simple way to prevent neural networks from overfitting. The journal of machine learning research, 15(1), 1929–1958.
- Unsupervised clickstream clustering for user behavior analysis. In Proceedings of the 2016 CHI conference on human factors in computing systems (pp. 225–236).
- Weak supervision for fake news detection via reinforcement learning. In Proceedings of the AAAI Conference on Artificial Intelligence, Volume 34(01) (pp. 516–523).
- Glad: group anomaly detection in social media analysis. ACM Transactions on Knowledge Discovery from Data (TKDD), 10(2), 1–22.
- Advances in variational inference. IEEE transactions on pattern analysis and machine intelligence, 41(8), 2008–2026.
- Vigdet: Knowledge informed neural temporal point process for coordination detection on social media. Advances in Neural Information Processing Systems, 34.
- A survey of fake news: Fundamental theories, detection methods, and opportunities. ACM Computing Surveys (CSUR), 53(5), 1–40.
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
Top Community Prompts
Collections
Sign up for free to add this paper to one or more collections.