A preliminary study of the feasibility of global evolutionary feature selection for big datasets under Apache Spark
Galar, M.; Triguero, I.; Bustince, H.; Herrera, F.
ISAAC TRIGUERO VELAZQUEZ I.TrigueroVelazquez@nottingham.ac.uk
Designing efficient learning models capable of dealing with tons of data has become a reality in the era of big data. However, the amount of available data is too much for traditional data mining techniques to be applicable. This issue is even more serious when evolutionary algorithms are a key part of the learning algorithm. In this scenario, one typical approach is to follow a divide-and-conquer strategy, where data is divided into different chunks that are individually and independently addressed. Afterwards, the partial knowledge obtained from each chunk of data is combined in order to give a solution to the problem. Nevertheless, these kinds of local approaches do not look at data as a whole, missing a global view of the problem, which may result in less accurate models that also depend on how data is split. In this work, we focus on evolutionary feature selection algorithms. A divide-and-conquer approach to handle evolutionary feature selection in big data was already developed. We aim at designing its global counterpart, which looks at the feature selection problem from a global perspective, making use of the data as a whole to select the most appropriate features. In order to do so, we consider Apache Spark as a big data technology where our algorithm is implemented. We design a genetic algorithm capable of dealing with big datasets by selecting the proper parameters for our base algorithm (the well-known CHC) and adapting the evaluation procedure to take all the distributed data into account. Several preliminary results are discussed to study the feasibility of global evolutionary feature selection methods for big datasets.
|Start Date||Jul 8, 2018|
|Publication Date||Jul 12, 2018|
|Publisher||Institute of Electrical and Electronics Engineers|
|Book Title||2018 IEEE Congress on Evolutionary Computation (CEC)|
|APA6 Citation||Galar, M., Triguero, I., Bustince, H., & Herrera, F. (2018). A preliminary study of the feasibility of global evolutionary feature selection for big datasets under Apache Spark. In 2018 IEEE Congress on Evolutionary Computation (CEC). , (1-8). https://doi.org/10.1109/CEC.2018.8477878|
|Additional Information||© 2018 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.|
Preliminary Study of the Feasibility of Global Evolutionary Feature Selection
You might also like
Redundancy and Complexity Metrics for Big Data Classification: Towards Smart Data
A Local Search with a Surrogate Assisted Option for Instance Reduction
Galaxy Image Classification Based on Citizen Science Data: A Comparative Study
Multigranulation Super-Trust Model for Attribute Reduction