Comparative Performance Analysis of Techniques for Automatic Drug Review Classification

Abstract

This study analyses the effectiveness of six text feature selection methods for automatic classification of drugreviews written in English using two different widely-known classifiers namely Support Vector Machines(SVM) and naïve Bayes (NB). In the study, a recently published public dataset namely Druglib includingdrug reviews in English was utilized in the experiments. For evaluation, Micro-F1 and Macro-F1 successmeasures were used. Also, 3-fold cross-validation is preferred to perform a fair evaluation. The featureselection methods used in the study are Distinguishing Feature Selector (DFS), Information Gain (IG), chi-square (CHI2), Discriminative Features Selection (DFSS), Improved Comprehensive Measurement FeatureSelection (ICMFS), and Relative Discrimination Criterion (RDC). However, experiments were performedusing two settings in which stemming was applied and not applied. Experiments indicated that ICMFS featureselection method is generally superior to the other feature selection methods according to the overall highestMicro-F1 and Macro-F1 scores achieved on drug reviews. While the highest Micro-F1 score was achievedwith the combination of NB classifier and ICMFS feature selection method, the highest Macro-F1 score wasachieved with the combination of NB classifier and DFSS feature selection method. The highest Micro-F1and Macro-F1 scores were achieved for the cases that stemming algorithm was not applied.