A temporal topic model for social trend prediction

Abstract

Social media provides increasing opportunities for users to voluntarily share their thoughts and concerns in a large volume of data. While user-generated data from each individual may not provide considerable information, when combined, they include variables which can convey significant events. In this thesis, we pursue the question of whether social media context can provide socio-behavior ``signals'' for scio-economic index prediction. The hypothesis is that crowd publicly available data in social media, in particular Twitter, may include predictive variables which can indicate the changes of socio-economic indexes. We developed content-based and user-centric prediction models where the objective is to employ Twitter content to predict whether the rates increase or decrease for the prospective time-frame. In order to collect Twitter data, we developed an activity-based sampling approach to collect credible users. The intention is to target users who are historically active rather than those who do not have enough contributions in the past. Since our problem has a sequential order, extracting meaningful patterns from historical tweets involves temporal analysis. Prediction models require to address information evolution, in which data are more related when they are close in time rather than further apart. We introduced a four-phase temporal topic detection model to infer predictive hidden topics. The model includes document partitioning, topic inference, topic selection, and document representation phases. In fact, a dynamic vocabulary is built to detect emerging topics. The extracted topics are compared over time to select more diverse and novel topics in each time consideration. The selected topics as predictive features are then applied in the proposed prediction models. The prediction models were evaluated for crime prediction in Chicago, Houston, San Francisco, and Philadelphia. The conducted experiments revealed the correlation between features extracted from the content and crime rates directions. The findings indicate that, extracted topics from content of active users achieved better performance compared to other features such as auxiliary ones. Overall, the proposed models in Twitter data collection and temporal topic detection have contributions in user-based sampling approaches and sequential topic detection problems, respectively.S