Visual and Predictive Analytics on Singapore News: Experiments on GDELT, Wikipedia, and ^STI

Abstract: The open-source Global Database of Events, Language, and Tone (GDELT) is the most comprehensive and updated Big Data source of important terms extracted from international news articles . We focus only on GDELT's Singapore events to better understand the data quality of its news articles, accuracy of its term extraction, and potential for prediction. To test news completeness and validity, we visually compared GDELT (Singapore news articles' terms from 1979 to 2013) to Wikipedia's timeline of Singaporean history. To test term extraction accuracy, we visually compared GDELT (CAMEO codes and TABARI system of extraction from Singapore news articles' text from April to December 2013) to SAS Text Miner's term and topic extraction. To perform predictive analytics, we propose a novel feature engineering method to transform row-level GDELT from articles to a user-specified temporal resolution. For example, we apply a decision tree using daily counts of feature values from GDELT to predict Singapore stock market's Straits Times Index (^STI). Of practical interest from the above results is SAS Visual Analytics' ability to highlight the various impacts of June 2013 Southeast Asian haze and December 2013 Little India riot on Singapore. Although Singapore is unique as a sovereign city-state, a leading financial centre, has strong international influence, and consists of a highly multi-cultural population, the visual and predictive analytics reported here are highly applicable to another country's GDELT data.