2013
Ferrer, Javier; Chicano, Francisco; Alba, Enrique
Estimating software testing complexity Journal Article
In: Information and Software Technology, vol. 55, no. 12, pp. 2125–2139, 2013.
Abstract | Links | BibTeX | Tags: branch coverage, Complexity, Correlations, ES, evolutionary algorithms, evolutionary testing, GA, Markov, search based software engineering, Testability, Validation
@article{Ferrer2013a,
title = {Estimating software testing complexity},
author = {Javier Ferrer and Francisco Chicano and Enrique Alba},
url = {http://www.sciencedirect.com/science/article/pii/S0950584913001535},
year = {2013},
date = {2013-01-01},
journal = {Information and Software Technology},
volume = {55},
number = {12},
pages = {2125--2139},
abstract = {CONTEXT Complexity measures provide us some information about software artifacts. A measure of the difficulty of testing a piece of code could be very useful to take control about the test phase. OBJECTIVE The aim in this paper is the definition of a new measure of the difficulty for a computer to generate test cases, we call it Branch Coverage Expectation (BCE). We also analyze the most common complexity measures and the most important features of a program. With this analysis we are trying to discover whether there exists a relationship between them and the code coverage of an automatically generated test suite. METHOD The definition of this measure is based on a Markov model of the program. This model is used not only to compute the BCE, but also to provide an estimation of the number of test cases needed to reach a given coverage level in the program. In order to check our proposal, we perform a theoretical validation and we carry out an empirical validation study using 2600 test programs. RESULTS The results show that the previously existing measures are not so useful to estimate the difficulty of testing a program, because they are not highly correlated with the code coverage. Our proposed measure is much more correlated with the code coverage than the existing complexity measures. CONCLUSION The high correlation of our measure with the code coverage suggests that the BCE measure is a very promising way of measuring the difficulty to automatically test a program. Our proposed measure is useful for predicting the behavior of an automatic test case generator.},
keywords = {branch coverage, Complexity, Correlations, ES, evolutionary algorithms, evolutionary testing, GA, Markov, search based software engineering, Testability, Validation},
pubstate = {published},
tppubtype = {article}
}
CONTEXT Complexity measures provide us some information about software artifacts. A measure of the difficulty of testing a piece of code could be very useful to take control about the test phase. OBJECTIVE The aim in this paper is the definition of a new measure of the difficulty for a computer to generate test cases, we call it Branch Coverage Expectation (BCE). We also analyze the most common complexity measures and the most important features of a program. With this analysis we are trying to discover whether there exists a relationship between them and the code coverage of an automatically generated test suite. METHOD The definition of this measure is based on a Markov model of the program. This model is used not only to compute the BCE, but also to provide an estimation of the number of test cases needed to reach a given coverage level in the program. In order to check our proposal, we perform a theoretical validation and we carry out an empirical validation study using 2600 test programs. RESULTS The results show that the previously existing measures are not so useful to estimate the difficulty of testing a program, because they are not highly correlated with the code coverage. Our proposed measure is much more correlated with the code coverage than the existing complexity measures. CONCLUSION The high correlation of our measure with the code coverage suggests that the BCE measure is a very promising way of measuring the difficulty to automatically test a program. Our proposed measure is useful for predicting the behavior of an automatic test case generator.