Validation of University Entrance Tests through Rasch Analysis

Initial preparation of first year university students is often assessed by means of an entrance examination, which tests their knowledge of mathematics and science as well as their reading skills. This paper illustrates how we used Rasch analysis to: (i) investigate the reliability and construct validity of a typical university entrance test administered in Italy; (ii) explore the extent to which mathematics, science, and reading items differ in their difficulty. Two studies were set up. In study 1, we analyzed the psychometric quality of an 80-item test administered in 2016 to N=2435 science and engineering freshmen. In study 2, we analyzed the responses of N=1223 students to a 100-item entrance test administered in 2017, 2018, and 2019 to an extended population of students. Results of both studies show that the analyzed entrance tests do not match unidimensional requirement, as proficiency in mathematics, science, and reading capability correspond to distinct latent traits. Moreover, items on different scientific topics have significantly different difficulty. The study shows a need for revising the analyzed tests to meet unidimensionality requirements. Moreover, the analysis of items’ difficulty suggests balancing in a more suitable way the difficulty of the different content areas.