• norsk
    • English
  • English 
    • norsk
    • English
  • Login
View Item 
  •   Home
  • Fakultet for informasjonsteknologi og elektroteknikk (IE)
  • Institutt for matematiske fag
  • View Item
  •   Home
  • Fakultet for informasjonsteknologi og elektroteknikk (IE)
  • Institutt for matematiske fag
  • View Item
JavaScript is disabled for your browser. Some features of this site may not work without it.

Time-series predictions with Recurrent Neural Networks - Studying Recurrent Neural Networks predictions and comparing to state-of-the-art Sequential Monte Carlo methods

Finsveen, Lene
Master thesis
Thumbnail
View/Open
19906_FULLTEXT.pdf (5.218Mb)
19906_COVER.pdf (1.556Mb)
URI
http://hdl.handle.net/11250/2563516
Date
2018
Metadata
Show full item record
Collections
  • Institutt for matematiske fag [1433]
Abstract
Recurrent Neural Networks (RNNs) have shown great success in sequence-to-sequence processing due to its ability to retain memory

while incrementally processing sequence elements.

It has become a fundamental algorithm for processing text and speech,

and is recently becoming more popular on time-series prediction as well.

Recent blog posts showing RNN flexibility applied to time-series prediction combined with

flexible and easy-to-use APIs such as Keras, are leading to a widespread use of RNN for time-series prediction.

The aim of this thesis is to study the performance of RNN to predict time-series under different scenarios of noise and stationarity.

In order to do this, we will simulate time-series from a state-space model (SSM) with known noise and stationarity parameters.

An SSM is a subclass of a Bayesian hierarchical model and simulates a hidden state

and observation depending only on static parameters and state. This is a common setup found in the signal processing literature.

The hidden state will act as an underlying signal while the observation is a mixture of signal and noise.

RNN models will try to predict the signal based on the simulated observed values.

We are going to experiment with a long short-term memory (LSTM) model,

which is the most popular type of RNNs currently used.

We evaluate the quality of the LSTM predictions by computing the cumulative mean square error (CMSE)

with respect to the true simulated signal.

In addition, we will estimate the signal using state-of-the-art Sequential Monte Carlo methods (SMC)

such as the Bootstrap filter and Particle Markov Chain Monte Carlo (PMCMC).

We then also compare the performance of the LSTM models with respect to the estimated signals.

The objective is to get insights into the LSTM performance level when compared to custom-tailored state-space models.

The general findings are that the LSTM models perform worse than custom-tailored SSM models in non-stationary scenarios.

PMCMC outperforms LSTM in all scenarios given the custom-tailored SSM parameters.

Bootstrap filtering struggles more with estimating the state in non-stationary scenarios as noise increases, but still perform better than LSTM.
Publisher
NTNU

Contact Us | Send Feedback

Privacy policy
DSpace software copyright © 2002-2019  DuraSpace

Service from  Unit
 

 

Browse

ArchiveCommunities & CollectionsBy Issue DateAuthorsTitlesSubjectsDocument TypesJournalsThis CollectionBy Issue DateAuthorsTitlesSubjectsDocument TypesJournals

My Account

Login

Statistics

View Usage Statistics

Contact Us | Send Feedback

Privacy policy
DSpace software copyright © 2002-2019  DuraSpace

Service from  Unit