Assessment and comparison of existing methods and datasets for sentiment analysis of Greek texts

Abstract

Sentiment Αnalysis is a well-known field of Natural Language Processing that is concerned with text classification. There is a vast number of papers, especially for the English language, that present state-of-the-art results on many different datasets using a variety of classification models. The aim of this work is to compare machine learning models on different datasets in both Greek and English. In order to achieve this aim, we used the well-known IMDb dataset from Stanford University, which is very often used for the evaluation of new text classification models, and one equivalent new dataset that we created in Greek from the Athinorama website. For our experiments, we used the following models: Logistic Regression, Support Vector Machine, Naïve Bayes, Decision Trees, XGBoost, Convolutional Neural Network, Long Short-Term Memory, Gated Recurrent Units, and Bidirectional Encoder Representations from Transformers (BERT). The first five models were combined with the TF-IDF vectorization technique, while the rest were combined with the Word Embeddings vectorization technique. The results show that the best classifier for sentiment analysis for both English and Greek is the pretrained BERT model. The difference in language does not seem to have a significant impact on the results, whereas the quality, the size, and the level of pre-processing of the data appear to play a significant role in the classification process. The reason we chose to deal with this work is the lack of research for the Greek language and our contribution is the Athinorama Light dataset that could play a significant role in future works for Greek language classification issues.