Automatic License Plate Recognition using Deep Learning Techniques

Abstract

Context: Automatic license plate recognition (ALPR) is used in many domains, such as parking, border control and motorway road tolling. Its importance has increased the recent years, with many new applications. High prediction accuracy and speed of ALPR is vital. Recent improvements in deep learning have increased its ability to solve complex visual recognition task. Using deep learning to improve the accuracy and speed of solving the ALPR task is for this reason promising.

Goal: The goal of this thesis is to propose a method using deep learning techniques solving the ALPR task and evaluate its processing speed and prediction accuracy.

Method: The research was divided into two stages. The first stage consisted of reviewing publications on the ALPR task, and reviewing and evaluating deep learning techniques, aiming at proposing a suitable method. The second stage consisted of evaluating the proposed method. The main criteria in the evaluation was processing speed and prediction accuracy. The dataset used to evaluate the prediction accuracy consisted of parked cars with mostly Norwegian license plates.

Results: A method using two object detection convolutional neural networks (CNNs) was proposed for the purpose of solving the ALPR task. The first network was retrained to detect license plates, and the second network to segment and recognize characters within the license plates detected by the first network. Both networks used the YOLOv2 architecture, as it was believed to be significantly faster and have slightly higher accuracy compared to the other reviewed architectures. To our knowledge, the proposed method is new and has never been evaluated on the ALPR task before. The evaluation of the proposed method resulted in an overall prediction accuracy of 97.6%, significantly outperforming other methods tested on the same dataset. The license plate detection achieved an accuracy of 99.8%, and the character detection an accuracy of 97.8%. When running the proposed method on a GPU, it used on average 17ms to perform plate detection on an image and on average 16ms to perform character detection on a plate, giving an overall processing time of 33ms per image. This is to our knowledge faster than all other methods described in the literature, even substantially faster than most of them. It is also fast enough to run on real-time video streams running in up to 30 FPS without the need to drop any frames.

Conclusion: This thesis is the first evaluation of a method using deep object detection CNNs to solve the ALPR task. The proposed method achieved very high prediction accuracy and outperformed all other methods considering processing speed when using a GPU. This suggests that using deep object detection CNNs is a good solution for the ALPR task.