RT Conference Proceedings
T1 Validating item response processes in digital competence assessment through eye-tracking techniques
A1 Bartolomé, Juan
A1 Garaizar, Pablo
A1 Bastida, Leire
A2 Garcia-Penalvo, Francisco Jose
AB This paper reports on an exploratory study with the aim to validate item response processes in digital competence assessment through eye-tracking techniques. When measuring complex cognitive constructs, it is crucial to correctly design the evaluation items to trigger the intended knowledge and skills. Furthermore, to assess the validity of a test requires considering not only the content of the evaluation tasks involved in the test, but also whether examinees respond to the tasks by engaging construct-relevant response processes. The eye tracking observations helped to fill an ‘explanatory gap’ by providing data on variation in item response processes that are not captured by other sources of process data such as think aloud protocols or computer-generated log files. We proposed a set of metrics that could help test designers to validate the different item formats used in the evaluation of digital competence. The gaze data provided detailed information on test item response strategies, enabling profiling of examinee engagement and response processes associated with successful performance. There were notable differences between the participants who correctly solved the tasks and those who failed, in terms of the time spent on solving them, as well as the data on their gazes. Moreover, this included insights into response processes which contributed to the validation of the assessment criteria of each item.
PB ACM
SN 978-1-4503-8850-4
SN 9781450388504
YR 2020
FD 2020-10-21
LA eng
NO Bartolomé , J , Garaizar , P & Bastida , L 2020 , Validating item response processes in digital competence assessment through eye-tracking techniques . in F J Garcia-Penalvo (ed.) , unknown . ACM International Conference Proceeding Series , ACM , pp. 738-746 , 8th International Conference on Technological Ecosystems for Enhancing Multiculturality, TEEM 2020 , Virtual, Online , Spain , 21/10/20 . https://doi.org/10.1145/3434780.3436641
NO conference
NO Publisher Copyright: © 2020 ACM.
DS TECNALIA Publications
RD 1 jul 2024