2013
Ferrer, Javier; Chicano, Francisco; Alba, Enrique
Estimating software testing complexity Journal Article
In: Information and Software Technology, vol. 55, no. 12, pp. 2125–2139, 2013.
Abstract | Links | BibTeX | Tags: branch coverage, Complexity, Correlations, ES, evolutionary algorithms, evolutionary testing, GA, Markov, search based software engineering, Testability, Validation
@article{Ferrer2013a,
title = {Estimating software testing complexity},
author = {Javier Ferrer and Francisco Chicano and Enrique Alba},
url = {http://www.sciencedirect.com/science/article/pii/S0950584913001535},
year = {2013},
date = {2013-01-01},
journal = {Information and Software Technology},
volume = {55},
number = {12},
pages = {2125--2139},
abstract = {CONTEXT Complexity measures provide us some information about software artifacts. A measure of the difficulty of testing a piece of code could be very useful to take control about the test phase. OBJECTIVE The aim in this paper is the definition of a new measure of the difficulty for a computer to generate test cases, we call it Branch Coverage Expectation (BCE). We also analyze the most common complexity measures and the most important features of a program. With this analysis we are trying to discover whether there exists a relationship between them and the code coverage of an automatically generated test suite. METHOD The definition of this measure is based on a Markov model of the program. This model is used not only to compute the BCE, but also to provide an estimation of the number of test cases needed to reach a given coverage level in the program. In order to check our proposal, we perform a theoretical validation and we carry out an empirical validation study using 2600 test programs. RESULTS The results show that the previously existing measures are not so useful to estimate the difficulty of testing a program, because they are not highly correlated with the code coverage. Our proposed measure is much more correlated with the code coverage than the existing complexity measures. CONCLUSION The high correlation of our measure with the code coverage suggests that the BCE measure is a very promising way of measuring the difficulty to automatically test a program. Our proposed measure is useful for predicting the behavior of an automatic test case generator.},
keywords = {branch coverage, Complexity, Correlations, ES, evolutionary algorithms, evolutionary testing, GA, Markov, search based software engineering, Testability, Validation},
pubstate = {published},
tppubtype = {article}
}
CONTEXT Complexity measures provide us some information about software artifacts. A measure of the difficulty of testing a piece of code could be very useful to take control about the test phase. OBJECTIVE The aim in this paper is the definition of a new measure of the difficulty for a computer to generate test cases, we call it Branch Coverage Expectation (BCE). We also analyze the most common complexity measures and the most important features of a program. With this analysis we are trying to discover whether there exists a relationship between them and the code coverage of an automatically generated test suite. METHOD The definition of this measure is based on a Markov model of the program. This model is used not only to compute the BCE, but also to provide an estimation of the number of test cases needed to reach a given coverage level in the program. In order to check our proposal, we perform a theoretical validation and we carry out an empirical validation study using 2600 test programs. RESULTS The results show that the previously existing measures are not so useful to estimate the difficulty of testing a program, because they are not highly correlated with the code coverage. Our proposed measure is much more correlated with the code coverage than the existing complexity measures. CONCLUSION The high correlation of our measure with the code coverage suggests that the BCE measure is a very promising way of measuring the difficulty to automatically test a program. Our proposed measure is useful for predicting the behavior of an automatic test case generator.
2012
Ferrer, Javier; Chicano, Francisco; Alba, Enrique
Evolutionary algorithms for the multi-objective test data generation problem Journal Article
In: Software: Practice and Experience, vol. 42, no. 11, pp. 1331–1362, 2012, ISSN: 00380644.
Abstract | Links | BibTeX | Tags: branch coverage, evolutionary algorithms, evolutionary testing, multi-objective test data generation, oracle cost, Search-based Software Engineering
@article{Ferrer2012a,
title = {Evolutionary algorithms for the multi-objective test data generation problem},
author = {Javier Ferrer and Francisco Chicano and Enrique Alba},
url = {http://doi.wiley.com/10.1002/spe.1135 http://arxiv.org/abs/1008.1900},
doi = {10.1002/spe.1135},
issn = {00380644},
year = {2012},
date = {2012-11-01},
journal = {Software: Practice and Experience},
volume = {42},
number = {11},
pages = {1331--1362},
abstract = {Cloud computing promises a radical shift in the provisioning of computing resource within the enterprise. This paper describes the challenges that decision makers face when assessing the feasibility of the adoption of cloud computing in their organisations, and describes our Cloud Adoption Toolkit, which has been developed to support this process. The toolkit provides a framework to support decision makers in identifying their concerns, and matching these concerns to appropriate tools/techniques that can be used to address them. Cost Modeling is the most mature tool in the toolkit, and this paper shows its effectiveness by demonstrating how practitioners can use it to examine the costs of deploying their IT systems on the cloud. The Cost Modeling tool is evaluated using a case study of an organization that is considering the migration of some of its IT systems to the cloud. The case study shows that running systems on the cloud using a traditional "always on" approach can be less cost effective, and the elastic nature of the cloud has to be used to reduce costs. Therefore, decision makers have to be able to model the variations in resource usage and their systems deployment options to obtain accurate cost estimates.},
keywords = {branch coverage, evolutionary algorithms, evolutionary testing, multi-objective test data generation, oracle cost, Search-based Software Engineering},
pubstate = {published},
tppubtype = {article}
}
Cloud computing promises a radical shift in the provisioning of computing resource within the enterprise. This paper describes the challenges that decision makers face when assessing the feasibility of the adoption of cloud computing in their organisations, and describes our Cloud Adoption Toolkit, which has been developed to support this process. The toolkit provides a framework to support decision makers in identifying their concerns, and matching these concerns to appropriate tools/techniques that can be used to address them. Cost Modeling is the most mature tool in the toolkit, and this paper shows its effectiveness by demonstrating how practitioners can use it to examine the costs of deploying their IT systems on the cloud. The Cost Modeling tool is evaluated using a case study of an organization that is considering the migration of some of its IT systems to the cloud. The case study shows that running systems on the cloud using a traditional "always on" approach can be less cost effective, and the elastic nature of the cloud has to be used to reduce costs. Therefore, decision makers have to be able to model the variations in resource usage and their systems deployment options to obtain accurate cost estimates.
2010
Ferrer, Javier; Chicano, Francisco; Alba, Enrique
Correlation Between Static Measures and Code Coverage in Evolutionary Test Data Generation. Journal Article
In: International Journal of Software Engineering and Its Applications, vol. 4, no. 4, pp. 57–78, 2010.
Links | BibTeX | Tags: branch coverage, evolutionary algorithms, evolutionary testing
@article{Ferrer2010,
title = {Correlation Between Static Measures and Code Coverage in Evolutionary Test Data Generation.},
author = {Javier Ferrer and Francisco Chicano and Enrique Alba},
url = {http://www.sersc.org/journals/IJSEIA/vol4_no4_2010/4.pdf http://www.earticle.net/Article.aspx?sn=147959 http://search.ebscohost.com/login.aspx?direct=true&profile=ehost&scope=site&authtype=crawler&jrnl=17389984&AN=61135648&h=W5TgdNvTb4pu8Avh8vLblbXFDPBwDb},
year = {2010},
date = {2010-01-01},
journal = {International Journal of Software Engineering and Its Applications},
volume = {4},
number = {4},
pages = {57--78},
keywords = {branch coverage, evolutionary algorithms, evolutionary testing},
pubstate = {published},
tppubtype = {article}
}
2009
Ferrer, Javier; Chicano, Francisco; Alba, Enrique
Correlation between static measures and code coverage in evolutionary test data generation Journal Article
In: International Journal of Software Engineering and its Applications, vol. 4, no. 1, pp. 57–79, 2009.
Links | BibTeX | Tags: branch coverage, Complexity, ES, evo, evolutionary algorithms, evolutionary testing, GA, McCabe, Metrics
@article{Ferrer2009,
title = {Correlation between static measures and code coverage in evolutionary test data generation},
author = {Javier Ferrer and Francisco Chicano and Enrique Alba},
url = {http://www.sistedes.es/TJISBD/Vol-3/No-1/articles/adis-09-Ferrer-Correlations.pdf http://www.cc.uah.es/drg/adis2009/articles/adis-09-Ferrer-Correlations.pdf http://www.sistedes.es/TJISBD/Vol-3/No-1/articles/adis-09-Ferrer-Correlations.pdf%5Cnhttp://www.cc},
year = {2009},
date = {2009-01-01},
journal = {International Journal of Software Engineering and its Applications},
volume = {4},
number = {1},
pages = {57--79},
keywords = {branch coverage, Complexity, ES, evo, evolutionary algorithms, evolutionary testing, GA, McCabe, Metrics},
pubstate = {published},
tppubtype = {article}
}
Ferrer, Javier; Chicano, Francisco; Alba, Enrique
On the correlation between static measures and code coverage using evolutionary test case generation Inproceedings
In: ADIS 2009 - Apoyo a la Decision en Ingenieria del Software, Evento Realizado en el Marco de las 14th Jornadas de Ingenieria del Software y Bases de Datos, JISBD 2009, 2009.
Abstract | BibTeX | Tags: branch coverage, Evo- lutionary strategy, evolutionary algorithms, evolutionary testing
@inproceedings{Ferrer2009b,
title = {On the correlation between static measures and code coverage using evolutionary test case generation},
author = {Javier Ferrer and Francisco Chicano and Enrique Alba},
year = {2009},
date = {2009-01-01},
booktitle = {ADIS 2009 - Apoyo a la Decision en Ingenieria del Software, Evento Realizado en el Marco de las 14th Jornadas de Ingenieria del Software y Bases de Datos, JISBD 2009},
volume = {3},
number = {1},
abstract = {Evolutionary testing is a very popular domain in the field of search based software engineering that consists in automatically generating test cases for a given piece of code using evolutionary algorithms. One of the most important measures used to evaluate the quality of the generated test suites is code coverage. In this paper we want to analyze if there exists a correlation between some static measures computed on the test program and the code coverage when an evolutionary test case generator is used. In particular, we use Evolutionary Strategies (ES) as search engine of the test case generator. We have also developed a program generator that is able to create Java programs with the desired values of the static measures. The experimental study includes a benchmark of 3600 programs automatically generated to find correlations between the measures. The results of this study can be used in future work for the development of a tool that decides the test case generation method according to the static measures computed on a given program.},
keywords = {branch coverage, Evo- lutionary strategy, evolutionary algorithms, evolutionary testing},
pubstate = {published},
tppubtype = {inproceedings}
}
Evolutionary testing is a very popular domain in the field of search based software engineering that consists in automatically generating test cases for a given piece of code using evolutionary algorithms. One of the most important measures used to evaluate the quality of the generated test suites is code coverage. In this paper we want to analyze if there exists a correlation between some static measures computed on the test program and the code coverage when an evolutionary test case generator is used. In particular, we use Evolutionary Strategies (ES) as search engine of the test case generator. We have also developed a program generator that is able to create Java programs with the desired values of the static measures. The experimental study includes a benchmark of 3600 programs automatically generated to find correlations between the measures. The results of this study can be used in future work for the development of a tool that decides the test case generation method according to the static measures computed on a given program.