How Does Incorporating the Response Times into Mixture Modelling Influence the Identification of Latent Classes for Mathematics Literacy Framework in PISA 2022?

Abstract

Based on PISA 2022 mathematics literacy test data for Türkiye, this study employed a mixture item response model to identify the ability-and non-ability latent classes of students. In line with the mixture item response theory modelling approach proposed by Jeon and De Boeck (2019), the relations between response times and item difficulty and success probabilities were examined by using four different models in a hierarchical comparison. The first of these models was a single-class two-parameter item response theory (2PL IRT) model (Model I), and the second one (Model II) was a two-class model called the ability class and the guessing class with a success probability fixed at 0.25. In the other two-class model (Model III), the success probability of the guessing class was freely estimated. The final model was a two-class model (Model IV) that included the ability class and the non-ability class, i.e. the one with the response time information as a covariate, in line with the approach proposed by Jeon and De Boeck (2019). As a result of the analysis, Model IV (a two-class model in which response time was included as a covariate) was found to be the best fitting model. Whereas the average item response times and success probabilities tended to be low in the non-ability class, these values were higher in the ability class. However, the ability class, which utilized time more effectively (with higher probability of success), was successful by responding rapidly to easy items while spending more time on difficult ones. As opposed to that, the overall low performance of the non-ability class was noteworthy since it turned out that their faster responses on easy items resulted in failure, whereas they were partially successful by dedicating more time to difficult ones. The latter group seems to have adopted a more superficial approach in which they used a type of item response strategy so that they could respond faster than the ability class on all items but tended to be careful by spending more time on difficult items.