Factors that influence the difficulty level of digital arithmetic assessment items for first-grade students

Abstract

Digital technologies enable new possibilities for the assessment of mathematical competence. When designing an assessment, it is essential to know how different design elements affect both the item difficulty and the strategies used by the children. In this paper, we investigate digital items that were designed to measure arithmetic competence as a component of the Foundational Number Sense (FoNS) framework for five-and six-year-old children. A Rasch analysis of the performance of 302 Norwegian children showed that the type of arithmetic problem and the magnitude of the answer strongly affected an item's difficulty level. Our qualitative observations indicated that certain additional design elements of the items might have influenced both the items' difficulty and the children's solution strategies. From a mixed methods perspective, we discuss the potential of different design elements to better assess children's understanding of numbers.