A Study on Flexible Bayes Classifier Using K-Nearest Neighbor Density Estimator

Naïve Bayes (NB) Classifier is considered one of the most widely used algorithms to solve classification problems due to its simplicity, high performance and fast computational process, which assumes that feature variables are independent and continuous variables follow gaussian distribution. Due to that most real-life applications may have feature variables that can’t be modeled by gaussian distribution, or may violate the independence assumption, Flexible Naïve Bayes (FNB) Classifier becomes more suitable, which estimates the densities using a non-parametric Kernel Density Estimator (KDE). In this paper, we introduce K-Nearest Neighbor Naïve Bayes (KNNB) Classifier, an improved flexible classifier which uses K-Nearest Neighbor Density Estimator (KNNDE) instead of KDE, such classifier would solve both NB problems, where the independence assumption is removed and the likelihood density is estimated for all feature variables together. Practical applications using R programming language on datasets from different fields were made for both FNB and KNNB classifiers with and without the independence assumption using 10-folds repeated cross validation, despite that FNB achieved greater accuracy than KNNB when dealing with the feature variables separately, the results showed that KNNB classifier performs better than FNB when the independence assumption is removed, meaning that KNNB is more efficient and suitable than FNB in real-life applications, where the independence assumption can be dropped.