• norsk
    • English
  • norsk 
    • norsk
    • English
  • Logg inn
Vis innførsel 
  •   Hjem
  • Øvrige samlinger
  • Publikasjoner fra CRIStin - NTNU
  • Vis innførsel
  •   Hjem
  • Øvrige samlinger
  • Publikasjoner fra CRIStin - NTNU
  • Vis innførsel
JavaScript is disabled for your browser. Some features of this site may not work without it.

A Configurable and Executable Model of Spark Streaming on Apache YARN

Lin, Jia-Chun; Lee, Ming-Chang; Yu, Ingrid Chieh; Johnsen, Einar Broch
Peer reviewed, Journal article
Accepted version
Thumbnail
Åpne
Lin (1.152Mb)
Permanent lenke
https://hdl.handle.net/11250/2731879
Utgivelsesdato
2020
Metadata
Vis full innførsel
Samlinger
  • Institutt for informasjonssikkerhet og kommunikasjonsteknologi [1641]
  • Publikasjoner fra CRIStin - NTNU [21889]
Originalversjon
International Journal of Grid and Utility Computing (IJGUC). 2020, 11 (2), 185-195.   10.1504/IJGUC.2020.105531
Sammendrag
Abstract: Streams of data are produced today at an unprecedented scale. Efficient and stable processing of these streams requires a careful interplay between the parameters of the streaming application and of the underlying stream processing framework. Today, finding these parameters happens by trial and error on the complex, deployed framework. This paper shows that high-level models can help to determine these parameters by predicting and comparing the performance of streaming applications running on stream processing frameworks with different configurations. To demonstrate this approach, this paper considers Spark Streaming, a widely used framework to leverage data streams on the fly and provide real-time stream processing. Technically, we develop a configurable and executable model to simulate both the streaming applications and the underlying Spark stream processing framework. Furthermore, we model the deployment of Spark Streaming on Apache YARN, which is a popular open-source distributed software framework for big data processing. We show that the developed model provides a satisfactory accuracy for predicting performance by means of empirical validation.
Utgiver
Inderscience
Tidsskrift
International Journal of Grid and Utility Computing (IJGUC)

Kontakt oss | Gi tilbakemelding

Personvernerklæring
DSpace software copyright © 2002-2019  DuraSpace

Levert av  Unit
 

 

Bla i

Hele arkivetDelarkiv og samlingerUtgivelsesdatoForfattereTitlerEmneordDokumenttyperTidsskrifterDenne samlingenUtgivelsesdatoForfattereTitlerEmneordDokumenttyperTidsskrifter

Min side

Logg inn

Statistikk

Besøksstatistikk

Kontakt oss | Gi tilbakemelding

Personvernerklæring
DSpace software copyright © 2002-2019  DuraSpace

Levert av  Unit