Uncertainty-Aware Autonomous Sensing with Deep Reinforcement Learnings

Abstract

The goal of many Internet of Things (IoT) sensing applications, such as environmental monitoring, is to support decision-making by providing valuable information about various phenomena. One approach to achieve this goal is to deploy a network of wireless sensors that collect and transmit measurements of the phenomena. However, typical wireless sensors have limited energy, computation, and communication resources, making it infeasible to measure continuously at a high rate. We can use efficient sensing techniques to address these resource constraints by selecting only the most informative measurements and avoiding wasting resources on redundant or least informative measurements. However, the design of such sensing techniques often involves manually fine-tuned heuristics or algorithms that are task-specific and non-adaptive. This approach is not scalable to the IoT setting, where a large number of sensors are deployed in dynamic and non-stationary environments. There is a need for more autonomous approaches to achieve scalable sensing solutions, where individual sensors autonomously learn strategies suitable for their specific environments and capabilities. Therefore, we investigate the use of deep reinforcement learning to develop autonomous sensing solutions. We first address the challenge of computational complexity by building a general-purpose simulator to train the behavior of sensing agents. We demonstrate the feasibility of using deep reinforcement learning to achieve autonomous sensing through three different use cases. We then explore using various reward functions, including utility-based, information-based, and value-based rewards, to align with application goals and guide learning toward desired behaviors. We finally show that explicitly representing our understanding about a phenomenon can guide the autonomous sensing agent in selecting informative measurements that maximally inform our prior knowledge of the phenomenon. Overall, the results of this work show that deep reinforcement learning can lead to increased autonomy and, thus, scalability in IoT sensing applications while achieving comparable performance to conventional methods.