LOF weighted knn regression ensemble and its application to a die manufacturing company

Abstract

K-nearest neighbor (KNN) algorithm is a widely used machine learning technique for prediction problems due its simplicity, flexibility and interpretability. When predicting the output variable of a data point, it basically averages the output values of its k closest neighbors. However, the impact of the neighboring points on the estimation may differ. Even though there are weighted versions of KNN, the effect of outliers and density differences within the neighborhoods are not considered. In order to fill this gap, we propose a novel weighting scheme for KNN regression based on local outlier factor (LOF). In particular, we combine the inverse of the Euclidean distance and LOF value so that the weights of the neighbors are determined using not only distance and connectivity but also outlier and density information around the neighborhood. Also, bootstrap aggregation is used to leverage the stability and accuracy of the LOF weighted KNN regression. Using real-life benchmark datasets, extensive experiments and statistical tests were performed for evaluating the performance of the proposed approach. The experimental results indicate the superior performance of the proposed approach in small neighborhood sizes. Moreover, the proposed approach was implemented in a make-to-order manufacturing company, and die production times were estimated successfully.