Skip to main content

Research Repository

Advanced Search

A preliminary study on hybrid spill-tree fuzzy k-nearest neighbors for big data classification

Maillo, Jesus; Luengo, Julián; Garcia, Salvador; Herrera, Francisco; Triguero, Isaac

Authors

Jesus Maillo

Julián Luengo

Salvador Garcia

Francisco Herrera



Abstract

The Fuzzy k Nearest Neighbor (Fuzzy kNN) classifier is well known for its effectiveness in supervised learning problems. kNN classifies by comparing new incoming examples with a similarity function using the samples of the training set. The fuzzy version of the kNN accounts for the underlying uncertainty in the class labels, and it is composed of two different stages. The first one is responsible for calculating the fuzzy membership degree for each sample of the problem in order to obtain smoother boundaries between classes. The second stage classifies similarly to the standard kNN algorithm but uses the previously calculated class membership degree. To deal with very large datasets, distributed versions of the Fuzzy kNN algorithm have been proposed. However, existing approaches remain not fully scalable as they aim to replicate the exact behavior of the Fuzzy kNN. In this work, we present an approximate and distributed Fuzzy kNN approach based on Hybrid Spill-Tree implemented under Apache Spark. The aim of this model is to alleviate the scalability problems and to deal with big datasets maintaining high accuracy. In our experiments, we compare in precision and runtime with the Fuzzy kNN for big data problems existing in the literature, running with datasets of up to 11 million instances. The results show an improvement in the runtime and accuracy with respect to the previous exact model.

Start Date Jul 8, 2018
Publication Date Jul 10, 2018
Publisher Institute of Electrical and Electronics Engineers
Pages 1-8
Book Title 2018 IEEE International Conference on Fuzzy Systems (FUXX-IEEE)
Chapter Number N/a
ISBN 9781509060207
APA6 Citation Maillo, J., Luengo, J., Garcia, S., Herrera, F., & Triguero, I. (2018). A preliminary study on hybrid spill-tree fuzzy k-nearest neighbors for big data classification. In 2018 IEEE International Conference on Fuzzy Systems (FUXX-IEEE). , (1-8). https://doi.org/10.1109/FUZZ-IEEE.2018.8491595
DOI https://doi.org/10.1109/FUZZ-IEEE.2018.8491595
Publisher URL https://ieeexplore.ieee.org/document/8491595
Additional Information © 2018 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.

Files





You might also like



Downloadable Citations

;