ACADEMIC RELATIONS

Empirical Evaluation of Automated Sentiment Analysis as a Decision Aid

ABSTRACT: This research is a part of a larger program on using text analytics techniques to increase the utility of textual content, especially in the social media and the Internet, for improved decision making. In the consumer decision making context, research has consistently shown that online word-of-mouth (WOM) plays an important role in shaping consumer attitudes and behaviors. Yet, despite their documented utility, explicit user scores, such as star ratings have limitations in certain contexts. Automatic sentiment analysis (SA), an analytics technique that assesses the “tone” of text, has been proposed as a way to deal with these shortcomings. These series of studies investigate the utility of SA tools as decision aids. In the first study, we compare two main techniques for analyzing sentiments in the context of consumer product reviews. Given that a noted gap in prior research has been the almost sole focus on short textual information that concerns specific contexts, we examine the role of contextual factors, namely review length and product category, on the accuracy of various sentiment analysis tools. Our results indicate that the lexicon-based approach to sentiment analysis outperforms machine-learning in almost all product contexts regardless of the length of the review. Machine learning had a better accuracy only in the case of long reviews about experience products and services. In the second study, we compare the results of the SA scores to consumer review ratings generated by human judges. In a third study, we propose to examine the effects of SA scores on decision making. Specifically, we develop a model that investigates the effects of review characteristics (length, extremity), product context (search vs. experience), and SA score representation on the utility of SA scores as measured by decision confidence, effort, and perceived usefulness.  

BIO: Ozgur Turetken (BSc., MBA, METU, Ph.D., Oklahoma State University) is a professor at, and the director of, Ryerson University’s Ted Rogers School of IT Management (Toronto, ON, Canada). He is currently transitioning to his role as the incoming Associate Dean for Research at Ted Rogers School of Management. Prior to Ryerson, he served on the faculty of Temple University’s Fox School of Business and Management. Dr. Turetken’s main research interests are in applied (text) analytics especially in the context of individual decision making. His research in this area has been funded by Natural Sciences and Engineering Research Council of Canada among other agencies, and has been published in international journals such as AIS Transactions on Human Computer Interaction, Communications of the ACM, ACM Database, Decision Support Systems, Information & Management, Information Systems Frontiers, and Information Systems and conferences such as ACM CIKM, AMCIS, ECIS, and ICIS. Ozgur is one of the former chairs of Association for Information Systems Special Interest Group on Decision Support and Analytics (AIS SIGDSA) for which he is now an elected advisory board member. He also serves in the HCI and analytics research communities as an editor and track chair.