Assessment of an expert judgment based human error probability study: A study of its validity, reliability and possible generalization

Abstract

Safetec has in cooperation with an oil company operating on the Norwegian continental shelf conducted an expert judgment based human error probability study where both human errors and violations were considered. This master thesis evaluates the validity, reliability and possible generalization of the human error probabilities (HEP) estimated by the experts. The reliability analysis has shown that the estimated HEP values show good overall reliability with some disagreement for certain operational errors. The analysis has been conducted by quantitatively and qualitatively looking at the standard deviation for the estimates to uncover the degree of agreement among the experts. The validity analysis was conducted by comparing the Safetec study with two established human reliability analysis methods, THERP and SPAR-H. This assessment looked at the similarities in numerical comparability and similarity in description between the established methods and the tasks evaluated in the Safetec study. From this study it was concluded that it is reason to believe that the Safetec study is based on good informed judgment and that it is in such a way valid. Lastly a study of the operational errors considered in the Safetec study resulted in 14 new generalized categories of HEP values. These new categories are meant to be applicable to more situations and industries.