17-03-2014, 10:24 PM
Abstract—There has been a number of software reliability
growth models (SRGMs) proposed in literature. Due to several
reasons, such as violation of models’ assumptions and
complexity of models, the practitioners face difficulties in
knowing which models to apply in practice. This paper
presents a comparative evaluation of traditional models and
use of genetic programming (GP) for modeling software
reliability growth based on weekly fault count data of three
different industrial projects. The motivation of using a GP
approach is its ability to evolve a model based entirely on prior
data without the need of making underlying assumptions. The
results show the strengths of using GP for predicting fault
count
Keywords-Software Reliabilty, Reliability Model, Metrics.
I. INTRODUCTION
A key element of software quality is software
reliability, defined as the ability of a system or component to
perform its required functions under stated conditions for a
specific period of time [13]. If the software frequently fails
to perform according to user-specified behavior, other
software quality factors matter less [19].
Software reliability growth modeling helps in deciding
project release time and managing project resources. After
the first software reliability growth model was proposed by
Jelinski and Moranda in 1972[14], there have been numerous
reliability growth models following it. The existence of a
large number of models requires a user to select and apply an
appropriate model. For practitioners, this may be an
unmanageable selection problem and there is a risk that the
selected model is unsuitable to the particulars of the project
in question. Some models are complex with many
parameters. Without extensive mathematical background,
practitioners cannot determine when it is applicable and
when the model diverges from reality. Moreover, these
parametric software reliability growth models are often
characterized by a number of assumptions [9] which are
necessary to develop a mathematical model. These
assumptions are often violated in real-world situations (see
e.g. [26]), therefore, causing problems in the long-term
applicability and validity of these models. Even if the
dynamics of the testing process are well known, there is no
guarantee that the model whose assumptions appear to best
suit these dynamics will be most appropriate [20].
Under this scenario, what becomes significantly
interesting is to have modeling mechanisms that can exclude
the pre-suppositions about the model and are based entirely
on the fault data. In this respect, genetic programming (GP)
could be used as an effective tool because, being a nonparametric
method, GP does not conceive a particular
structure for the resulting model and GP also does not take
any assumptions about the distribution of the data.
In an earlier study, we studied about the suitability of
using GP for building software reliability growth models [2].
In this paper, we present the results of comparison between
models evolved using GP and three other traditional SRGMs
based on weekly fault count data of three projects carried out
by a large telecommunication company. We compare the
models using measures of model validity, goodness of fit and
residual analysis. The comparative results indicate that in
terms of model validity, two out of three measures favored
GP evolved models. The GP evolved model also represented
comparatively better good- ness of fit, while residual
analysis showed that the predictions from the GP evolved
model are comparatively less b iased.
growth models (SRGMs) proposed in literature. Due to several
reasons, such as violation of models’ assumptions and
complexity of models, the practitioners face difficulties in
knowing which models to apply in practice. This paper
presents a comparative evaluation of traditional models and
use of genetic programming (GP) for modeling software
reliability growth based on weekly fault count data of three
different industrial projects. The motivation of using a GP
approach is its ability to evolve a model based entirely on prior
data without the need of making underlying assumptions. The
results show the strengths of using GP for predicting fault
count
Keywords-Software Reliabilty, Reliability Model, Metrics.
I. INTRODUCTION
A key element of software quality is software
reliability, defined as the ability of a system or component to
perform its required functions under stated conditions for a
specific period of time [13]. If the software frequently fails
to perform according to user-specified behavior, other
software quality factors matter less [19].
Software reliability growth modeling helps in deciding
project release time and managing project resources. After
the first software reliability growth model was proposed by
Jelinski and Moranda in 1972[14], there have been numerous
reliability growth models following it. The existence of a
large number of models requires a user to select and apply an
appropriate model. For practitioners, this may be an
unmanageable selection problem and there is a risk that the
selected model is unsuitable to the particulars of the project
in question. Some models are complex with many
parameters. Without extensive mathematical background,
practitioners cannot determine when it is applicable and
when the model diverges from reality. Moreover, these
parametric software reliability growth models are often
characterized by a number of assumptions [9] which are
necessary to develop a mathematical model. These
assumptions are often violated in real-world situations (see
e.g. [26]), therefore, causing problems in the long-term
applicability and validity of these models. Even if the
dynamics of the testing process are well known, there is no
guarantee that the model whose assumptions appear to best
suit these dynamics will be most appropriate [20].
Under this scenario, what becomes significantly
interesting is to have modeling mechanisms that can exclude
the pre-suppositions about the model and are based entirely
on the fault data. In this respect, genetic programming (GP)
could be used as an effective tool because, being a nonparametric
method, GP does not conceive a particular
structure for the resulting model and GP also does not take
any assumptions about the distribution of the data.
In an earlier study, we studied about the suitability of
using GP for building software reliability growth models [2].
In this paper, we present the results of comparison between
models evolved using GP and three other traditional SRGMs
based on weekly fault count data of three projects carried out
by a large telecommunication company. We compare the
models using measures of model validity, goodness of fit and
residual analysis. The comparative results indicate that in
terms of model validity, two out of three measures favored
GP evolved models. The GP evolved model also represented
comparatively better good- ness of fit, while residual
analysis showed that the predictions from the GP evolved
model are comparatively less b iased.