Using Sensing Technologies to Explain Children’s Self-Representation in Motion-Based Educational Games

Abstract

Motion-Based Touchless Games (MBTG) are being investigated as a promising interaction paradigm in children's learning experiences. Within these games, children's digital persona (i.e, avatar), enables them to efficiently communicate their motion-based interactivity. However, the role of children's Avatar Self-Representation (ASR) in educational MBTG is rather under-explored. We present an in-situ within subjects study where 46 children, aged 8--12, played three MBTG with different ASRs. Each avatar had varying visual similarity and movement congruity (synchronisation of movement in digital and physical spaces) to the child. We automatically and continuously monitored children's experiences using sensing technology (eye-trackers, facial video, wristband data, and Kinect skeleton data). This allowed us to understand how children experience the different ASRs, by providing insights into their affective and behavioural processes. The results showed that ASRs have an effect on children's stress, arousal, fatigue, movement, visual inspection (focus) and cognitive load. By exploring the relationship between children's degree of self-representation and their affective and behavioural states, our findings help shape the design of future educational MBTG for children, and emphasises the need for additional studies to investigate how ASRs impacts children's behavioural, interaction, cognitive and learning processes.