A practical tutorial on bagging and boosting based ensembles for machine learning: Algorithms, software tools, performance study, practical perspectives and opportunities

Simple item page
dc.contributor.authorGonzález, Sergio
dc.contributor.authorGarcía, Salvador
dc.contributor.authorDel Ser, Javier
dc.contributor.authorRokach, Lior
dc.contributor.authorHerrera, Francisco
dc.contributor.institutionIA
dc.date.issued2020-12
dc.descriptionPublisher Copyright: © 2020 Elsevier B.V.
dc.description.abstractEnsembles, especially ensembles of decision trees, are one of the most popular and successful techniques in machine learning. Recently, the number of ensemble-based proposals has grown steadily. Therefore, it is necessary to identify which are the appropriate algorithms for a certain problem. In this paper, we aim to help practitioners to choose the best ensemble technique according to their problem characteristics and their workflow. To do so, we revise the most renowned bagging and boosting algorithms and their software tools. These ensembles are described in detail within their variants and improvements available in the literature. Their online-available software tools are reviewed attending to the implemented versions and features. They are categorized according to their supported programming languages and computing paradigms. The performance of 14 different bagging and boosting based ensembles, including XGBoost, LightGBM and Random Forest, is empirically analyzed in terms of predictive capability and efficiency. This comparison is done under the same software environment with 76 different classification tasks. Their predictive capabilities are evaluated with a wide variety of scenarios, such as standard multi-class problems, scenarios with categorical features and big size data. The efficiency of these methods is analyzed with considerably large data-sets. Several practical perspectives and opportunities are also exposed for ensemble learning.en
dc.description.sponsorshipThis work was supported by the Spanish National Research ProjectTIN2017-89517-P, and by a research scholarship (FPU) given to the author Sergio González by the Spanish Ministry of Education, Culture and Sports. Javier Del Ser would also like to thank the Basque Government for its funding support through the ELKARTEK and EMAITEK programs, as well as the Consolidated Research Group MATHMODE (IT1294-19) granted by the Department of Education of this institution. CatBoost is available as an open-source library of the company Yandex LLC [119] . CatBoost library is not as popular as XGBoost or LightGBM libraries and lacks some software supports. Table 11 gathers the programming language and computation paradigms supported by the CatBoost project.
dc.description.statusPeer reviewed
dc.format.extent33
dc.identifier.citationGonzález , S , García , S , Del Ser , J , Rokach , L & Herrera , F 2020 , ' A practical tutorial on bagging and boosting based ensembles for machine learning : Algorithms, software tools, performance study, practical perspectives and opportunities ' , Information Fusion , vol. 64 , pp. 205-237 . https://doi.org/10.1016/j.inffus.2020.07.007
dc.identifier.doi10.1016/j.inffus.2020.07.007
dc.identifier.issn1566-2535
dc.identifier.urlhttp://www.scopus.com/inward/record.url?scp=85089214531&partnerID=8YFLogxK
dc.language.isoeng
dc.relation.ispartofInformation Fusion
dc.relation.projectIDSpanish National Research ProjectTIN2017-89517-P
dc.relation.projectIDU.S. Department of Education, ED
dc.relation.projectIDEusko Jaurlaritza, IT1294-19
dc.relation.projectIDMinisterio de Educación, Cultura y Deporte, MECD
dc.rightsinfo:eu-repo/semantics/restrictedAccess
dc.subject.keywordsClassification
dc.subject.keywordsDecision trees
dc.subject.keywordsEnsemble learning
dc.subject.keywordsMachine learning
dc.subject.keywordsSoftware
dc.subject.keywordsSoftware
dc.subject.keywordsSignal Processing
dc.subject.keywordsInformation Systems
dc.subject.keywordsHardware and Architecture
dc.titleA practical tutorial on bagging and boosting based ensembles for machine learning: Algorithms, software tools, performance study, practical perspectives and opportunitiesen
dc.typejournal article
Files