Applying temporal dependence to detect changes in streaming data

Abstract

Detection of changes in streaming data is an important mining task, with a wide range of real-life ap- plications. Numerous algorithms have been proposed to efficiently detect changes in streaming data. However, the limitation of existing algorithms is that they as- sume that data are generated independently. In partic- ular, temporal dependencies of data in a stream are still not thoroughly studied. Motivated by this, in this work we propose a new efficient method to detect changes in streaming data by exploring the temporal dependencies of data in the stream. As part of this, we introduce a new statistical model called the candidate change point (CCP) model, with which the main idea is to compute the probabilities of finding change points in the stream. The computed probabilities are used to generate a dis- tribution, which is, in turn, used in statistical hypoth- esis tests to determine the candidate changes. We use the CCP model to develop a new algorithm called Can- didate Change Point Detector (CCPD), which detects change points in linear time, and is thus applicable for real-time applications. Our extensive experimental eval- uation demonstrates the efficiency and the feasibility of our approach.