A Weighted Similarity Measure for k-Nearest Neighbors Algorithm

Abstract

One of the most important problems in machine learning, which has gained importance in recent years, isclassification. The k-nearest neighbors (kNN) algorithm is widely used in classification problem becauseit is a simple and effective method. However, there are several factors affecting the performance of kNNalgorithm. One of them is determining an appropriate proximity (distance or similarity) measure.Although the Euclidean distance is often used as a proximity measure in the application of the kNN,studies show that the use of different proximity measures can improve the performance of the kNN. In thisstudy, we propose the Weighted Similarity k-Nearest Neighbors algorithm (WS-kNN) which use aweighted similarity as proximity measure in the kNN algorithm. Firstly, it calculates the weight of eachattribute and similarity between the instances in the dataset. And then, it weights similarities by attributeweights and creates a weighted similarity matrix to use as proximity measure. The proposed algorithm iscompared with the classical kNN method based on the Euclidean distance. To verify the performance ofour algorithm, experiments are made on 10 different real-life datasets from the UCI (UC Irvine MachineLearning Repository) by classification accuracy. Experimental results show that the proposed WS-kNNalgorithm can achieve comparative classification accuracy. For some datasets, this new algorithm giveshighly good results.