On-the-fly Detection of Autogenerated Tweets

Abstract: Most previous work related to tweet classification have focused on identifying a given tweet as a spam, or to classify a Twitter user account as a spammer or a bot. In most cases the tweet classification has taken place offline, on a pre-collected dataset of tweets. In this paper we present an \emph{on-the-fly} approach to classify each newly downloaded tweet as \emph{autogenerated} or not. We define an autogenerated tweet (AGT) as a tweet where all or parts of the natural language content is generated automatically by a bot or other type of program. Our on-the-fly approach makes use of two classifiers. The first classifies a tweet solely based on the twitter text and the tweet metadata that comes with every tweet. It is used for tweets posted by unknown users with no available tweet history. An unknown user also triggers a batch job to start downloading the missing user timeline information. The second classifier is used for tweets posted by a user where the user timeline is downloaded and available. Initially, it will be the first classifier that handles most of the tweets. This will gradually change and after an initialization phase where we download historic data for the most active users, we reach a state where the second classifier handles a vast majority of all the tweets. A simulation using our on-the-fly detection mechanism indicates that we can handle Twitter streams with up to 68,000 unique users each day. The bottleneck is the time required to download new user timelines. The AGT detection is very accurate. In a set of 5,000 tweets we correctly classified about 98\% of all AGTs using a subject-wise cross-validation.