Skip to main content

Research Repository

Advanced Search

A preliminary study of the feasibility of global evolutionary feature selection for big datasets under Apache Spark

Galar, M.; Triguero, I.; Bustince, H.; Herrera, F.


M. Galar

H. Bustince

F. Herrera


Designing efficient learning models capable of dealing with tons of data has become a reality in the era of big data. However, the amount of available data is too much for traditional data mining techniques to be applicable. This issue is even more serious when evolutionary algorithms are a key part of the learning algorithm. In this scenario, one typical approach is to follow a divide-and-conquer strategy, where data is divided into different chunks that are individually and independently addressed. Afterwards, the partial knowledge obtained from each chunk of data is combined in order to give a solution to the problem. Nevertheless, these kinds of local approaches do not look at data as a whole, missing a global view of the problem, which may result in less accurate models that also depend on how data is split. In this work, we focus on evolutionary feature selection algorithms. A divide-and-conquer approach to handle evolutionary feature selection in big data was already developed. We aim at designing its global counterpart, which looks at the feature selection problem from a global perspective, making use of the data as a whole to select the most appropriate features. In order to do so, we consider Apache Spark as a big data technology where our algorithm is implemented. We design a genetic algorithm capable of dealing with big datasets by selecting the proper parameters for our base algorithm (the well-known CHC) and adapting the evaluation procedure to take all the distributed data into account. Several preliminary results are discussed to study the feasibility of global evolutionary feature selection methods for big datasets.


Galar, M., Triguero, I., Bustince, H., & Herrera, F. (2018). A preliminary study of the feasibility of global evolutionary feature selection for big datasets under Apache Spark. In 2018 IEEE Congress on Evolutionary Computation (CEC). , (1-8).

Conference Name 2018 IEEE Congress on Evolutionary Computation (CEC)
Start Date Jul 8, 2018
End Date Jul 13, 2018
Acceptance Date Mar 15, 2018
Online Publication Date Oct 4, 2018
Publication Date Jul 12, 2018
Deposit Date Oct 18, 2018
Publicly Available Date Oct 18, 2018
Publisher Institute of Electrical and Electronics Engineers
Pages 1-8
Book Title 2018 IEEE Congress on Evolutionary Computation (CEC)
Chapter Number N/a
ISBN 9781509060177
Public URL
Publisher URL
Additional Information © 2018 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.


Preliminary Study of the Feasibility of Global Evolutionary Feature Selection (477 Kb)

You might also like

Downloadable Citations