Transformer models for Greek and English language understanding

Abstract

This thesis examines the effectiveness of Transformer models in natural lan- guage processing (NLP), focusing on text classification and sentiment analysis for both Greek and English texts. It marks a significant advancement in NLP by demonstrating how Transformers, through their self-attention mechanisms, offer a superior approach to understanding context and relationships within language com- pared to traditional RNN architectures.Through a detailed exploration of several Transformer models, such as BERT, RoBERTa, and GPT-2, and their comparison with RNNs, this study evaluates their performance across different languages and text types. Fine-tuning language-specific pretrained versions of these models on two distinct datasets—Greek product reviews and English tweets related to the #metoo movement—allows for an examination of the adaptability of Transformers to diverse linguistic challenges. The research ultimately illustrates that Transformer models significantly outperform traditional RNNs in sentiment analysis, highlighting their considerable potential to enhance language understanding across varied linguistic and cultural contexts.