Transparency and Explainability in Financial Data Science

Abstract

Financial data science has experienced rapid developments in recent years with the expansion of ever-growing data at an exponential rate. The proponents of data science argue that data science techniques will dominate and improve many domains of science in the next decades. However, several critics and concerns remain about its widespread adoption in the financial field due to the absence of transparency and explainability in the current generation of data science techniques. For instance, researchers’ limited access to valuable data restrict the scientific developments and their benefits to both the academic community and industries. Additionally, while some state-of-the-art data science techniques, such as deep neural networks, have high prediction accuracy, they have been criticized for being black box methods that allow limited transparency into the decision process. This thesis contributes to increasing transparency and explainability in financial data science by solving three types of research problems in energy and financial credit markets. First, my results shed more transparency on the intraday electricity trading by showing the impact of renewable energies on trader’s strategies. In particular, I focus on the impact of wind and photovoltaic infeed on intraday electricity pricing. This study is particularly relevant to increasing transparency in intraday trading, since updates in weather forecasting errors are typically unavailable to researchers. Second, I employ state-of-the-art deep neural networks to price day-ahead electricity related to market coupling and use a post-hoc explainability technique to interpret prediction results. Third, I propose a data-driven explainable case-based reasoning method to predict financial credit risk, and show the relevance of its explainability in prediction results. For intraday market traders, this thesis sheds light on how updated forecasts of renewable energies influence traders‘ behavior in the intraday trading. Moreover, it benefits intraday traders by proposing ways to model renewable energies forecasts that will further enhance existing econometric models for intraday electricity prices. Further, this thesis provides an efficient hybrid deep neural networks framework to predict day-ahead electricity prices under the consideration of market coupling for day-ahead electricity market participants. A post-hoc explainability technique is used to interpret the importance of the feature inputs, demand/supply variables, which offers more information and knowledge for cross-border market regulators and traders to move towards an integrated electricity market in Europe. Last, this thesis shows that financial institutions can benefit from the explainable case-based reasoning system to better serve their customers and reduce financial risk, in line with regulatory requirements. Compared with other machine learning methods, the proposed method provides superior prediction results of financial risk and has a major relevance to the decision-making. This allows banks and other financial institutions to not only correctly map the probability of default for any borrower, but also to explain the underlying reason for default. In addition, results are highly relevant to borrowers, as it provides suggestions on how to improve their financial status to obtain new credit.