The Role of Theoretical Models in Ecologically Valid Studies: the example of a video Quality of Experience model

Abstract

This paper discusses the problem of ecological va-lidity of Quality of Experience experiments in a broader context of internal and external validity. We argue that the issue of trade-off between control in the experiment and generalization of the results requires diverse experimental protocols. To enable this process comparability of experiments has to be guaran-teed. To improve comparability and communication between researchers, we propose using theoretical models as tools for clearly communicating the assumptions underlying subjective experiments. These models describe the relationships between variables and enable comparability of research. We demonstrate how the theoretical model of video QoE can be used for the comparison of three independent QoE studies. In conclusion, we stress that ecological validity is a mean to an extended external validity. We suggest that new experimental protocols that increase the number of Influence Factors (IFs) measured have to provide comparability with other experiments. Theoretical models can facilitate this process and stimulate higher comparability between studies in QoE subdomains. A community effort is needed to adjust the theoretical model to new use cases, particularly with new immersive multimedia. This paper inspires further research on ecological validity in QoE and encourages the adoption of theoretical models for more comparable research designs.