A comparative review of regression ensembles on drug design datasets

Kaynakça (1)

1. name samples features features 1 benzo 195 32 32 2 carbolenes 37 1142 15 3 chang 34 1142 7 4 cristalli 32 1142 14 5 depreux 26 1142 12 6 mtp 274 1142 24 7 pah 80 112 10 8 pdgfr 79 320 11 9 phen 22 110 6 10 phenetyl 22 628 7 11 qsbr y 15 9 3 12 qsfsr 19 9 3 13 selwood 31 53 5 14 strupcz 34 1142 15 15 yokohoma 12 1142 11 Experimental results Seven base regressors were used together with each ensemble algorithm on 15 regression-type drug design problems. The experiments were done to answer the following questions in drug design problems: - Do the algorithm ensembles generate more successful results than a single algorithm? - What is the most successful ensemble algorithm? - What is the base algorithm–ensemble pair with the best results? - Which algorithm performs well with the ensembles? - What is the most successful single algorithm? - How are the algorithms and datasets grouped according to their performances? - How does the dimension reduction process affect the results? To answer these questions, 36 algorithms ((4 ensemble + 1 single) × (7 base algorithms) + Zero Rule algorithm = 36) were employed on the 15 drug design datasets described in Table 3 and their dimensionally reduced versions. A 5 × 2 cross validation was used and the RMSE results were averaged. The RMSE is defined as: RM SE a lg.name = 1 N N i=1 y i a lg.name − y i actual 2, (1) where y i a lg.name is the prediction of alg.name for the ith test sample, y i actual is the actual output value of the ith test sample, and N is the number of test samples. The Zero Rule algorithm measures the default error of a dataset. The RMSE value of the Zero Rule is calculated as follows: RM SE ZeroRule = 1 N N i=1 y m − y i actual 2, y m = 1 T T j=1 y j actual, (2) where y j actual is the actual output value of the jth training sample, y i a lg.name is the prediction of alg.name for the ith test sample, T is the number of training samples, and N is the number of test samples. Our base learners and ensemble algorithms have some hyperparameters to optimize. We used 2-fold cross-validation to optimize these parameters. In bagging, we optimized the bagging size by trying values of 50%, 75%, and 100%. In additive regression, we optimized the shrinkage by trying values of 0.1, 0.5, and 1. In random subspace, we optimized the subspace size by trying values of 25%, 50%, and 75%. In rotation forest, we optimized the remove percentage by trying values of 25%, 50%, and 75%. In the M5P and REP trees, we optimized th