Utilisation of OLAP Technology on Large Data Volumes in the Traffic Data Domain

Abstract

In this report we investigate the performance of OLAP usage in a traffic data environment and propose possible improvements to an existing solution. The traffic data consists of 3 billion location and time measurements. These historical data are used in analysing future traffic volumes in specific time frames and locations, as well as building reports that aggregates data over time. We are primarily interested in measurements describing volume and velocity. The existing solution is static. It only supports simple pre-aggregations of the traffic volume. If users request more detailed information and reports based on velocity, the server has to read through the entire data set (3 billion) and aggregate on the fly. We create OLAP cubes that make the data, including the aggregates available. These cubes already contain much of the aggregate information, and when clients request reports, the cube can return the answers without reading the raw data. We implement the whole process, from extracting the raw data to building different cubes, in order to represent and retrieve aggregated data efficiently. This has been done using software from both Oracle and Microsoft. In the report we discuss possible solutions and give reasons for our choice of implementation. At the end of the report we perform performance measurements to compare the different implementations -- Oracle, Microsoft and the already existing solution.