Browsing by Keyword "Benchmarking"
Now showing 1 - 5 of 5
Results Per Page
Sort Options
Item The District Energy-Efficient Retrofitting of Torrelago (Laguna de Duero – Spain)(2019-06-21) Vasallo, A; Vallejo, E; Massa, G; Macía, A; Pablos, L; Criado, C; Arrizabalaga, E; Iturralde, J; Gordaliza, A; De Castro, I; Larrinaga, F; PLANIFICACIÓN ENERGÉTICA; SISTEMAS TÉRMICOS EFICIENTESThe urban growth is estimated to reach up the 66 % by 2050 and consequently the need of resources within the cities will increase significantly. This, combined with the 40 % of energy consumption and 36 % of CO2 emissions of the building sector, makes necessary to accelerate the transition towards more sustainable cities. The CITyFiED project contributes to this transition, aiming to develop an innovative and holistic methodological approach for energy-efficient district renovation and deliver three large scale demonstration cases in the cities of Lund (Sweden), Laguna de Duero (Spain) and Soma (Turkey). CITyFiED methodology consists of several phases that ease the decision-making tasks towards the district renovation, considering the energy efficiency as the main pillar and local authorities as clients. For the case of Torrelago district (Spain) the intervention consists of a set of energy conservative measures including the facąde retrofitting of 143.025 m2 of living space in 31 twelve-storey buildings; the renovation of the district heating network with a new biomass thermal plant; the integration of renewable energy sources, including a micro-cogeneration system, and the installation of individual smart meters. After the renovation action, one-year monitoring campaign is ongoing. The CITyFiED monitoring platform will collect information from the energy systems and deliver environmental, technical, economic and social key performance indicators by March 2019. At the end of the project the achievement of the predefined goals will be verified: Up to 36 % of energy saving and 3,429 tons-CO2/yr emissions saving covering the 59,4 % of the energy consumption with renewable sources.Item Making Bipedal Robot Experiments Reproducible and Comparable: The Eurobench Software Approach: The Eurobench Software Approach(2022-08-29) Remazeilles, Anthony; Dominguez, Alfonso; Barralon, Pierre; Torres-Pardo, Adriana; Pinto, David; Aller, Felix; Mombaur, Katja; Conti, Roberto; Saccares, Lorenzo; Thorsteinsson, Freygardur; Prinsen, Erik; Cantón, Alberto; Castilla, Javier; Sanz-Morère, Clara B.; Tornero, Jesús; Torricelli, Diego; Tecnalia Research & Innovation; Robótica Médica; Medical TechnologiesThis study describes the software methodology designed for systematic benchmarking of bipedal systems through the computation of performance indicators from data collected during an experimentation stage. Under the umbrella of the European project Eurobench, we collected approximately 30 protocols with related testbeds and scoring algorithms, aiming at characterizing the performances of humanoids, exoskeletons, and/or prosthesis under different conditions. The main challenge addressed in this study concerns the standardization of the scoring process to permit a systematic benchmark of the experiments. The complexity of this process is mainly due to the lack of consistency in how to store and organize experimental data, how to define the input and output of benchmarking algorithms, and how to implement these algorithms. We propose a simple but efficient methodology for preparing scoring algorithms, to ensure reproducibility and replicability of results. This methodology mainly constrains the interface of the software and enables the engineer to develop his/her metric in his/her favorite language. Continuous integration and deployment tools are then used to verify the replicability of the software and to generate an executable instance independent of the language through dockerization. This article presents this methodology and points at all the metrics and documentation repositories designed with this policy in Eurobench. Applying this approach to other protocols and metrics would ease the reproduction, replication, and comparison of experiments.Item More is not Always Better: Insights from a Massive Comparison of Meta-heuristic Algorithms over Real-Parameter Optimization Problems(Institute of Electrical and Electronics Engineers Inc., 2021) Del Ser, Javier; Osaba, Eneko; Martinez, Aritz D.; Bilbao, Miren Nekane; Poyatos, Javier; Molina, Daniel; Herrera, Francisco; IA; QuantumMuch controversy has been lately risen around the design and performance of modern bio-inspired optimization methods, in particular due to the alleged lack of algorithmic novelty in their definition with respect to traditional heuristic solvers. In this work we present a first attempt at shedding empirical evidence over this debate, for which results of a benchmark with unprecedented scales in terms of problems and algorithms are reported and discussed. Specifically, informed conclusions are held in what refers to the claimed superior performance of these bio-inspired solvers and their competitiveness when compared to competition-winning alternatives. Finally, we prove that the tailored selection of a subset of problems and techniques can unfairly bias the comparisons favoring any of such algorithms, ultimately arriving at illusory conclusions about their comparative performance.Item A multi-cloud framework for measuring and describing performance aspects of cloud services across different application types(SciTePress, 2014) Kousiouris, G.; Giammatteo, G.; Evangelinou, A.; Galante, N.; Kevani, E.; Stampoltas, C.; Menychtas, A.; Kopaneli, A.; Ramasamy Balraj, K.; Kyriazis, D.; Varvarigou, T.; Stuer, P.; Orue-Echevarria Arrieta, L.; Tecnalia Research & InnovationCloud services have emerged as an innovative IT provisioning model in the recent years. However, after their usage severe considerations have emerged with regard to their varying performance due to multitenancy and resource sharing issues. These issues make it very difficult to provide any kind of performance estimation during application design or deployment time. The aim of this paper is to present a mechanism and process for measuring the performance of various Cloud services and describing this information in machine understandable format. The framework is responsible for organizing the execution and can support multiple Cloud providers. Furthermore we present approaches for measuring service performance with the usage of specialized metrics for ranking the services according to a weighted combination of cost, performance and workload.Item A prescription of methodological guidelines for comparing bio-inspired optimization algorithms(2021-12) LaTorre, Antonio; Molina, Daniel; Osaba, Eneko; Poyatos, Javier; Del Ser, Javier; Herrera, Francisco; Quantum; IABio-inspired optimization (including Evolutionary Computation and Swarm Intelligence) is a growing research topic with many competitive bio-inspired algorithms being proposed every year. In such an active area, preparing a successful proposal of a new bio-inspired algorithm is not an easy task. Given the maturity of this research field, proposing a new optimization technique with innovative elements is no longer enough. Apart from the novelty, results reported by the authors should be proven to achieve a significant advance over previous outcomes from the state of the art. Unfortunately, not all new proposals deal with this requirement properly. Some of them fail to select appropriate benchmarks or reference algorithms to compare with. In other cases, the validation process carried out is not defined in a principled way (or is even not done at all). Consequently, the significance of the results presented in such studies cannot be guaranteed. In this work we review several recommendations in the literature and propose methodological guidelines to prepare a successful proposal, taking all these issues into account. We expect these guidelines to be useful not only for authors, but also for reviewers and editors along their assessment of new contributions to the field.