Comparing metrics to evaluate performance of regression methods for decoding of neural signals

No Thumbnail Available
Identifiers
URI: https://hdl.handle.net/11556/2144
ISSN: 1557-170X
ISBN: 9781424492718
DOI: 10.1109/EMBC.2015.7318553
Publication date
2015-11-04
Authors
Advisors
Journal Title
Journal ISSN
Volume Title
Publisher
Institute of Electrical and Electronics Engineers Inc.
Citations
Google Scholar
Export
Research Projects
Organizational Units
Journal Issue
Abstract
The use of regression methods for decoding of neural signals has become popular, with its main applications in the field of Brain-Machine Interfaces (BMIs) for control of prosthetic devices or in the area of Brain-Computer Interfaces (BCIs) for cursor control. When new methods for decoding are being developed or the parameters for existing methods should be optimized to increase performance, a metric is needed that gives an accurate estimate of the prediction error. In this paper, we evaluate different performance metrics regarding their robustness for assessing prediction errors. Using simulated data, we show that different kinds of prediction error (noise, scaling error, bias) have different effects on the different metrics and evaluate which methods are best to assess the overall prediction error, as well as the individual types of error. Based on the obtained results we can conclude that the most commonly used metrics correlation coefficient (CC) and normalized root-mean-squared error (NRMSE) are well suited for evaluation of cross-validated results, but should not be used as sole criterion for cross-subject or cross-session evaluations.
Description
Publisher Copyright: © 2015 IEEE.
Keywords
Signal Processing , Biomedical Engineering , Computer Vision and Pattern Recognition , Health Informatics
Type
conference output
Citation
Spuler , M , Sarasola-Sanz , A , Birbaumer , N , Rosenstiel , W & Ramos-Murguialday , A 2015 , Comparing metrics to evaluate performance of regression methods for decoding of neural signals . in 2015 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBC 2015 . , 7318553 , Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBS , vol. 2015-November , Institute of Electrical and Electronics Engineers Inc. , pp. 1083-1086 , 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBC 2015 , Milan , Italy , 25/08/15 . https://doi.org/10.1109/EMBC.2015.7318553
conference