Exploiting available data sources for ex-post evaluation of railway projects: Case illustrations with traffic and mobile phone data

Abstract

Technological developments increase the amount of available data sources in all industries. This thesis focuses on how we can exploit these data sources in ex-post project evaluation to support evaluators with new insight. The focus has been on railways and railway projects.

A project should be evaluated to establish to what degree the intended effects on the users and the society are achieved. The ex-post evaluation should be carried out a few years after major infrastructure projects are completed. One matter that should be evaluated is tactical success, which is measured in terms of effectiveness. Effectiveness measures if the project achieved its goals and is typically assessed based on the change of state before and after. However, evaluators experience problems in getting hold of essential data on the pre- and post-situation. For the evaluators, data collected from the railway operations are mainly available as aggregated numbers from performance and evaluation reports.

Typical goals of railway projects are to reduce travel time and increase capacity and demand. Two measures of particular interest are therefore punctuality and traffic volume. Punctuality is whether the traffic runs according to the timetable, and statistical numbers on punctuality are well-developed measures. However, a deeper evaluation of punctuality can be provided by analysing delay propagation, and that has been more difficult to obtain a good measure on. The number of travellers is an important performance measure that has been difficult for evaluators to obtain good data on, often because train operators consider such data confidential business information. In addition, there is a challenge regarding varying quality and coverage of the available data on the number of travellers.

Data of relevance to ex-post project evaluation are generated and collected from both the construction phase and the operation phase. The data collected from railway operations are useful when evaluating the tactical and strategic success of a project. The conclusions drawn from this evaluation can provide useful insight and learning to the strategic planning and concept development of future projects. The author has provided two practical examples of how data generated and collected during railway operations can be exploited to obtain relevant information for the evaluation. This was done through thorough investigation into two measures of particular interest for ex-post evaluation of railway projects, i.e. delay propagation and number of travellers.

Delay propagation was analysed based on traffic data, and a method was developed to find cases of knock-on delay on single tracks. The tool allows an attempt at indicating the direction of knock-on effects. In addition, the method traces the propagation of delay from one train to the next to find the networks of dynamic delay propagations. Mobile phone data were investigated as an alternative source of the number of travellers. The possibility of using mobile phone data is interesting because it is independent of the railway operators. The study showed that it is possible to combine mobile phone data with railway infrastructure and train traffic data. The findings show a potential for utilising mobile phone data to collect the number of travellers on the railway.

The data collected from technologies used during the construction project are useful when evaluating the project execution. This includes the experience report, which is the internal form of knowledge sharing, and the evaluation of efficiency, which examines time, cost and quality. These evaluations are important for learning from similar projects during the construction phase and can contribute to faster decisions.

Ex-post project evaluation has traditionally been qualitative, but because of technological developments, more data from the railway operations are available. It is still a problem to get hold of data, especially when it is about private information, while other data sources have become more available. The two empirical studies are good examples of how such data can provide useful insight into the ex-post evaluation of railway projects.