Skyline Computing over multiple Data Streams with a Storm Cluster.

Abstract

Skyline computing has received considerable attention over the last decade. The skyline of a multidimensional point set, consist of interesting points which are not dominated by any other point within base set. Lately the attention has especially been related to computing over data streams. This thesis targets multiple horizontal split streams, using horizontal and vertical scaling provided by Storm, a stream processing framework for cluster. Skylines are incrementally computed when data from streams are received. When multiple layers of skylines are utilized, every increment of a skyline is send onto next layer for further computing. The last layer will contain the final skyline set. Different Storm topologies were proposed, implemented and tested. A discussion of significant observations and an overall conclusion is presented. Some adoptions and optimizations were made towards algorithm completeness to suit an imaged stock exchange setting with limited resources available for a stock trader.