Προσέγγιση δυαδικής ταξινόμησης για τον εντοπισμό περιπτώσεων παιδικής εκμετάλλευσης σε μηνύματα συνομιλίας με χρήση της πλατφόρμας SPARK

Abstract

The present thesis challenges the probability of sexual offenders identification in chat messages using machine learning algorithms. Text filtering and pre-processing techniques are emphasized and tests with various classification algorithms of Apache Spark’s MlLib library will be presented. For the implementation, a large-scale data analysis engine (SPARK) was used in collaboration with an HDFS Hadoop Cluster. The dataset used due to the nature of the problem is quite unbalanced so emphasis is placed on the selection of the most qualified evaluation metric to balance the so-called “algorithm bias” towards the weak class.