Realizing Context-Aware Services through Intelligent Mobile Data Analysis

Abstract

In recent years there has been a rapid development of mobile technologies, Internet of Things (IoT) and cellular network infrastructures. In combination with the fast evolution of data analysis, and particularly within machine learning, this has led to new unprecedented opportunities for building smart environments. Central to smart environments lie context-aware systems. These systems sense the users environment, analyse the sensed data to deduce new insights and, from this, provide content, experiences, help and suggestions to the users. Accessing and providing such services have never been easier, since more than 6.5 billion people in the world own smartphones, a platform prime for supporting context-aware services. A topical form of context-aware services, when using smartphones as sensing equipment, are so-called location-aware services. Their particular characteristic is the utilization of the user’s location to provide useful suggestions related to their surroundings. Location-aware services, in particular, services used in retail (shopping) and public transport, are the main focus of this thesis. That are two domains, people generally interact with on a daily basis. In retail and shopping, so-called mobile recommender systems (MRS) are used to perform recommendations such as points of interest or products in the user’s proximity. Crucial to these services are the accurate real-time locations of both, the user and the products they suggest. The location of the user can be provided using a real-time location system (RTLS) tracking the user’s smartphone. However, an accurate, efficient solution to the location of the product is not as readily available. In public transport on the other hand, the detection of the mobile context of vehicles and their passengers is key to realize intelligent transportation systems (ITS). A topical example of this is the in-vehicle presence of a passenger, essential for context-aware services such as automated-ticketing. In order for these systems to work, the accuracy of the solution needs to be incredibly high, something, the state-of-the-art solutions available today still lack. To support these next-generation location-aware services, advanced context reasoning techniques, i.e., algorithms extracting useful information from the data sensed from the user’s environment, are of utmost importance. Machine learning algorithms form a very promising category of such contextreasoning techniques. In consequence, most of the work done in this Ph.D. project centers on them. Particularly, machine learning technology is used in this thesis to address challenges regarding context-aware services within the above mentioned fields retail and public transport. Altogether, we provided the following four contributions: The first contribution introduced in this thesis, is an automatic product localisation algorithm. The product locator proposed in this thesis infers the location of the products in a store by accumulating the locations at which customers stop when picking up products as well as the list of purchased products. A simulation-based environment shows that 99.9% out of 8, 000 products in a typical large Norwegian grocery store can be correctly located by aggregating data from customers over a 12-day period. The second contribution presented in this thesis is DeepMatch, a highly accurate in-vehicle presence detection algorithm. The approach is utilizing the smartphone of a passenger to analyze and match the sensor event streams of the device against the streams of sensors embedded in an on-board reference unit installed in public transportation vehicles. The matching is facilitated by a new deep learning model employed in a distributed fashion, where the feature extraction and dimensionality reduction is offloaded to the smartphones and the reference unit, while the matching is performed on a remote server. The approach achieved an invehicle prediction accuracy of 0.9781 on a dataset consisting of real data gathered by volunteers. The third contribution builds on the second one. It consists of the method DeepMatch2, the successor of Deepmatch, that increases its accuracy from 0.9781 to 0.9851. In addition, the algorithm improves its efficiency, effectively reducing the amount of data needed by the model by a factor of four. Furthermore, we propose a travelling user inference system based on Deep- Match2 with the ability to infer if and for which period of time a passenger makes a trip in a public transport vehicle with a very low error rate. The fourth contribution of this thesis is Ataraxis, a solution to hardwareless in-vehicle presence prediction. Through the collaboration with Public Transportation Authorities, we learned that some PTAs do not have the control to decide autonomously about the hardware that is installed in the vehicles they use. Therefore, the hardware-based solution proposed in Deep- Match and DeepMatch2, i.e., the installation of additional hardware in the vehicles, is not always suitable. Ataraxis addresses this challenge. A deep convolutional neural network to detect the transport mode of users from the sensor events generated by ordinary smartphones was developed. The user mode is used in combination with a GPS trace of the user and nearby public transport vehicles in order to infer the in-vehicle presence of the user. The deep learning model created for Ataraxis achieved an F1 Score of 98.69% when classifying the four user modes driving a car, riding a bike walking and using public transport. To summarize, in this thesis, we contribute applied research through several learning algorithms, system designs, and software solutions in order to enhance and improve the intelligence and quality of location-based contextaware services within retail and public transport. Furthermore, we show through extensive empirical experiments that the proposed approaches can be used in practice without negatively impacting the users smartphones. It answered the main research objective as well as provided several business critical patents to the industrial partner.