| dc.description.abstract | Transformer-based neural network architectures have recently demonstrated state-of-the-art performance in many Natural Language Processing tasks. Furthermore, there have been successful attempts to apply the same principles to problems beyond the scope of NLP. A particularly interesting application area is time series forecasting as there is a great potential for improving the current techniques and the research to date has rarely considered Transformer-based architectures for forecasting. However, manual design and optimization of neural network architectures and their hyperparameters has proven to be difficult, time-consuming and mainly driven by trial and error.
This work explores the use of evolutionary computation to design Transformer-based architectures suitable for time series forecasting. The proposed neural architecture search system is capable of performing an automated evolution-driven search to determine the optimal architectural components as well as their parameterization and internal structure. The performance of the evolved architectures is assessed by experiments which compare the achieved forecasting accuracy with accuracies of common forecasting methods. A selection of time series benchmarks is used as a base for the comparison. The main contributions consist of the mentioned system and the final discovered architecture. The work also introduces a genetic representation for evolving Transformer-based architectures which can be seen as another contribution. | |
