Multiobjective evolutionary pruning of Deep Neural Networks with Transfer Learning for improving their performance and robustness

Simple item page
dc.contributor.authorPoyatos, Javier
dc.contributor.authorMolina, Daniel
dc.contributor.authorMartínez-Seras, Aitor
dc.contributor.authorDel Ser, Javier
dc.contributor.authorHerrera, Francisco
dc.contributor.institutionIA
dc.date.accessioned2024-07-24T12:09:16Z
dc.date.available2024-07-24T12:09:16Z
dc.date.issued2023-11
dc.descriptionPublisher Copyright: © 2023
dc.description.abstractEvolutionary Computation algorithms have been used to solve optimization problems in relation with architectural, hyper-parameter or training configuration, forging the field known today as Neural Architecture Search. These algorithms have been combined with other techniques such as the pruning of Neural Networks, which reduces the complexity of the network, and the Transfer Learning, which lets the import of knowledge from another problem related to the one at hand. The usage of several criteria to evaluate the quality of the evolutionary proposals is also a common case, in which the performance and complexity of the network are the most used criteria. This work proposes MO-EvoPruneDeepTL, a multi-objective evolutionary pruning algorithm. MO-EvoPruneDeepTL uses Transfer Learning to adapt the last layers of Deep Neural Networks, by replacing them with sparse layers evolved by a genetic algorithm, which guides the evolution based in the performance, complexity and robustness of the network, being the robustness a great quality indicator for the evolved models. We carry out different experiments with several datasets to assess the benefits of our proposal. Results show that our proposal achieves promising results in all the objectives, and direct relation are presented among them. The experiments also show that the most influential neurons help us explain which parts of the input images are the most relevant for the prediction of the pruned neural network. Lastly, by virtue of the diversity within the Pareto front of pruning patterns produced by the proposal, it is shown that an ensemble of differently pruned models improves the overall performance and robustness of the trained networks.en
dc.description.sponsorshipF. Herrera, D. Molina and J. Poyatos are supported by the R&D and Innovation project with reference PID2020-119478GB-I00 granted by the Spain’s Ministry of Science and Innovation and European Regional Development Fund (ERDF) . A. Martinez-Seras and J. Del Ser would like to thank the Basque Government, Spain for the funding support received through the EMAITEK and ELKARTEK programs, as well as the Consolidated Research Group MATHMODE ( IT1456-22 ) granted by the Department of Education of this institution .
dc.description.statusPeer reviewed
dc.identifier.citationPoyatos , J , Molina , D , Martínez-Seras , A , Del Ser , J & Herrera , F 2023 , ' Multiobjective evolutionary pruning of Deep Neural Networks with Transfer Learning for improving their performance and robustness ' , Applied Soft Computing Journal , vol. 147 , 110757 . https://doi.org/10.1016/j.asoc.2023.110757
dc.identifier.doi10.1016/j.asoc.2023.110757
dc.identifier.issn1568-4946
dc.identifier.urihttps://hdl.handle.net/11556/3957
dc.identifier.urlhttp://www.scopus.com/inward/record.url?scp=85171623378&partnerID=8YFLogxK
dc.language.isoeng
dc.relation.ispartofApplied Soft Computing Journal
dc.relation.projectIDBasque Government, Spain, IT1456-22
dc.relation.projectIDR&D and Innovation, PID2020-119478GB-I00
dc.relation.projectIDU.S. Department of Education, ED
dc.relation.projectIDMinisterio de Ciencia e Innovación, MICINN
dc.relation.projectIDEuropean Regional Development Fund, ERDF
dc.rightsinfo:eu-repo/semantics/openAccess
dc.subject.keywordsEvolutionary Deep Learning
dc.subject.keywordsMulti-objective algorithms
dc.subject.keywordsOut of Distribution detection
dc.subject.keywordsPruning
dc.subject.keywordsTransfer Learning
dc.subject.keywordsSoftware
dc.titleMultiobjective evolutionary pruning of Deep Neural Networks with Transfer Learning for improving their performance and robustnessen
dc.typejournal article
Files