Code Coverage and Software Reliability

Abstract

With an ever-growing competition among software vendors in supplying customers with tailored, high-quality systems, an emphasis is put on creating products that are well-tested and reliable. During the last decade and a half numerous articles have been published that deal with code coverage and its effect, whether existent or not, on reliability. The last few years have also witnessed an increasing number of software tools for automating the data collection and presentation of code coverage information for applications being tested. In this report we aim at presenting available and frequently used measures of code coverage, the practical applications and typical misconceptions of code coverage and its role in software development nowadays. Then we take a look at the notion of reliability in computer systems and which elements that constitute a software reliability model. With the basics of code coverage and reliability estimation in place, we try to assess the status of the relationship between code coverage and reliability, highlight the arguments for and against its existence and briefly survey a few proposed models for connecting code coverage to reliability. Finally, we examine an open-source tool for automated code coverage analysis, focusing on its implementation of the coverage measures it supports, before assessing the feasibility of integrating a proposed approach for reliability estimation into this software utility.