A Simulator for Evaluating Smart Home Activity Recognition with RGB-D Cameras: A comparison between traditional smart home sensors and the Kinect

Abstract

In this thesis we will look at how the Kinect-camera can improve activity recognition and behavioral pattern discovery in a smart home. Well researched algorithms available in the Activity Learner-software allow for us to perform recognition-tests and pattern discovery with ease, but we need datasets that include camera data to allow comparisons to be made. In order to do this a simulator is designed that features an autonomous agent performing activities in a virtual smart home by using methods inspired by automated planning, existing simulators and the life simulation game The Sims .A model is proposed for using the Kinect as a sensor compatible with these existing algorithms, and the simulator features virtual cameras that output pose information based on what we can reasonably expect in a real life setting by looking at previous research. We will see that the camera can detect activities reliably in many cases, making the case that the modeled output is applicable to real life settings. Also the camera offers more detailed positioning possibilities, making it simple to get the location of residents in the environment.Our results indicate that using advanced cameras like the Kinect can not only reliably improve recognition accuracy, but also reduce the amount of required sensors in the environment. The simulator is capable of generating large datasets, and is easily customizable to simulate additional activities or other environments.There are still many issues to address before implementing a Kinect-enabled smart home in real life, but these findings indicate that while work on activity recognition has come a long way using only regular sensors, the increased reliability of computer vision can provide a significant boost. This boost can hopefully in turn lead to more reliable applications that leverage smart home monitoring, like for instance assisting or monitoring elderly residents living by themselves.