Investigating the Effects of User Features in Hate Speech Detection on Twitter

Abstract

Detecting hate speech has become an increasingly important task for online communities. Current methods for handling such unintended content are often heavily dependent on manual effort, and are therefore not scalable or efficient enough considering the large and growing corpus of user-generated data. Automatic hate speech detection is a challenging task, and a majority of the research in the field is targeting the task through the use of text. However, despite the emerging scientific studies to address the problem, there is still a need for further efforts to improve the quality and efficiency of detection methods. This motivates for research on how non-textual features can be utilized to enhance detection performance.

By using data from Twitter, the aim of this study was to investigate the potential effects user features have on hate speech classification. An overview of characteristic traits and promising user features was established through a study of related literature, and a quantitative analysis of Twitter datasets was conducted to better understand the characteristics of users, based on their tweets. The findings from the analysis show that there is not necessarily any correlations of hateful text and characteristics of the users who had posted them. Through experiments with a baseline hate speech classifier on three different datasets, it was found that combining certain user features with textual features slightly improved classification performance, than only using textual features. While the incorporation of user features resulted in varying impact on performance for the different datasets used, network-related features provided consistent improvement of the model performance. The inclusion of specific features was also found to be detrimental to the classifier performance, and some features only proved their usefulness when used in combination with others. These findings combine to suggest that there is a potential for incorporating user features to improve performance of hate speech detection models, but that more research should explore how they are most effectively utilized.