Constructing Sentiment Lexicon with Game for Annotation Collection

Radosky, L., Blstak, M.

Abstract: While research of sentiment analysis became very popular on the global scope, in Slovak language as an under-resourced language there are still many issues to be tackled, especially the lack of resources. In this paper, we introduce a sentiment analysis game designed to collect sentiment annotations. The game is intended for a single player who, motivated by game score, chooses the sentiment category of each word of a sentence. We describe the annotation collection process during which over 12 500 annotations of individual words and over 1 000 annotations of entire sentences were obtained within a week. The collected annotations were used to construct a sentiment lexicon. Using artificial bee colony algorithm, optimal lexicon construction parameters were discovered. To evaluate the final lexicon’s usefulness, we applied simple sentence-level lexicon-based sentiment analysis methods on a manually annotated dataset of mobile phone reviews. The same was done with other existing lexicons for comparison. The results of our experiments show that collecting annotations using our game can be useful as a method for constructing a sentiment lexicon.

Cite: Radoský L., Blšták M. (2021) Constructing Sentiment Lexicon with Game for Annotation Collection. In: Espinosa-Anke L., Martín-Vide C., Spasić I. (eds) Statistical Language and Speech Processing. SLSP 2021. Lecture Notes in Computer Science, vol 13062. Springer, Cham. https://doi.org/10.1007/978-3-030-89579-2_4