Semi-Supervised Modeling and Detection of Changes in Topic Sentiment in Near Real Time Using Social Media Data

Abstract:

While real time polling of topical opinions can occur, on election day for example; this process is rare, expensive and limited in scope. Data collection of social media posts, however, is not expensive and users freely share their opinions on a wide range of topics. While twitter users are not a representative sample of the population, detecting changes in sentiment or opinion of a topic in this population is less subject to bias. This research addresses several challenges and problems in this space. As a proof of concept, we will start with a labeled dataset of posts from political pundits. Our model will learn topics in common with pundits from both sides of the aisle, and will subsequently learn to classify posts as sympathetic to either democrat or republican. Traditional sentiment analysis is limited here; our model will not be looking for posts reflective of positive or negative feelings about a topic, but rather whether the author of the post is expressing an opinion that is left or right leaning. We will then show how this model can be used to track changes in popularity of a particular point of view over time. Finally we will show how this model can be used in near real time to discern the changing views of the general population who have not identified their political leaning.