REAL-TIME NETWORK INTRUSIONPREVENTION

Abstract

It is not economically or technically feasible to make complex computersystems that are completely secure. New attacks are constantly developedby attackers and the security situation can therefore rapidly change. In orderto detect and stop attackers before any damage is done, automated toolshave to be deployed because there is not enough time for manual intervention.Therefore there is a need for online risk assessment and proactive defensemechanisms like Intrusion Prevention System (IPS). In the area of computersecurity there have been only a few quantitative security measures until now,and there are few published cases for methods and tools based on such measures.The main areas of this thesis are: Quantitative characterization of riskand security in computer systems or networks; and dynamic risk and securityassessment based on network monitoring. During our research, the focus hasbeen narrowed down to look for answers to the following problems:Is it possible (and practical) to reuse some of the stochastic modelingtechniques used to model dependable systems?Can Hidden Markov Models (HMMs) be successfully used in real timerisk assessment?Is it feasible to prevent attacks against systems and networks based onrisk assessment?For these problems a Markov model describing the interaction between thesystem and attackers in a quantitative manner is proposed. The Markov modeldescribes the different security states of a network, and the transitions betweenthem.Based on the initial Markov model, a HMM modeling the trustworthinessof sensors collecting security relevant information in a computer network isproposed. The sensor model is used for online risk assessment based on observationsfrom sensors in a network. A security measure called intrusionfrequency is used. The intrusion frequency is estimated from the state distributionestimated by the HMM. The sensor model has been validated throughsimulations, and through experiment with synthetic and real network traffic.

Two different approaches to online risk assessment are proposed: one basedon costs associated width states and one based on a hierarchical fuzzy inferencesystem. Three different methods for aggregation of alerts from multiplenetwork sensors are discussed. The first method was to use the average of therisk estimated by each sensor, this solution have some obvious drawbacks e.g.when the risk from two sensors are aggregated where one is very trustworthyand one is very little trustworthy, in this case we would have been better offusing only the risk from the most trustworthy sensor instead of the average.The second method produces a minimum variance estimator of the risk. Thissolution is based on a strict assumption on independence between sensors. Inthe third proposal, one common distribution over the security state space ismaintained. The distribution is updated when an observation is received, usingthe sensors of the corresponding HMM. The fine tuning of the fuzzy logicbased risk assessment is achieved using a neural network learning technique. ADistributed Intrusion Prevention System (DIPS) architecture based on fuzzyonline risk assessment is presented as a practical application of the modelsdeveloped in thesis.