Assessment of class mutation operators for C++ with the MuCPP mutation system

Pedro Delgado-Pérez*, Inmaculada Medina-Bulo, Francisco Palomo-Lozano, Antonio García-Domínguez, Juan José Domínguez-Jiménez

*Corresponding author for this work

Research output: Contribution to journalArticle

Abstract

Context: Mutation testing has been mainly analyzed regarding traditional mutation operators involving structured programming constructs common in mainstream languages, but mutations at the class level have not been assessed to the same extent. This fact is noteworthy in the case of C++, despite being one of the most relevant languages including object-oriented features. Objective: This paper provides a complete evaluation of class operators for the C++ programming language. MuCPP, a new system devoted to the application of mutation testing to this language, was developed to this end. This mutation system implements class mutation operators in a robust way, dealing with the inherent complexity of the language. Method: MuCPP generates the mutants by traversing the abstract syntax tree of each translation unit with the Clang API, and stores mutants as branches in the Git version control system. The tool is able to detect duplicate mutants, avoid system headers, and drive the compilation process. Then, MuCPP is used to conduct experiments with several open-source C++ programs. Results: The improvement rules listed in this paper to reduce unproductive class mutants have a significant impact in the computational cost of the technique. We also calculate the quantity and distribution of mutants generated with class operators, which generate far fewer mutants than their traditional counterparts. Conclusions: We show that the tests accompanying these programs cannot detect faults related to particular object-oriented features of C++. In order to increase the mutation score, we create new test scenarios to kill the surviving class mutants for all the applications. The results confirm that, while traditional mutation operators are still needed, class operators can complement them and help testers further improve the test suite.

Original languageEnglish
Pages (from-to)169-184
Number of pages16
JournalInformation and Software Technology
Volume81
DOIs
Publication statusPublished - 1 Jan 2017

Fingerprint

Structured programming
Object oriented programming
Testing
Application programming interfaces (API)
Computer programming languages
Control systems
Costs
Experiments

Bibliographical note

© 2016, Elsevier. Licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International http://creativecommons.org/licenses/by-nc-nd/4.0/

Corrigendum: Delgado-Pérez, P., Medina-Bulo, I., Palomo-Lozano, F., García-Domínguez, A., & Domínguez-Jiménez, J. J. (2016). Corrigendum to 'Assessment of class mutation operators for C++ with the MuCPP mutation system' [Information and Software Technology, 81, (2017) 169-184]. Information and Software Technology. DOI: 10.1016/j.infsof.2017.02.001

Keywords

  • C++
  • Class mutation operators
  • Mutation system
  • Mutation testing
  • Object-oriented programming

Cite this

Delgado-Pérez, Pedro ; Medina-Bulo, Inmaculada ; Palomo-Lozano, Francisco ; García-Domínguez, Antonio ; Domínguez-Jiménez, Juan José. / Assessment of class mutation operators for C++ with the MuCPP mutation system. In: Information and Software Technology. 2017 ; Vol. 81. pp. 169-184.
@article{a6d69feaa54342728ce3e7d2b11339ea,
title = "Assessment of class mutation operators for C++ with the MuCPP mutation system",
abstract = "Context: Mutation testing has been mainly analyzed regarding traditional mutation operators involving structured programming constructs common in mainstream languages, but mutations at the class level have not been assessed to the same extent. This fact is noteworthy in the case of C++, despite being one of the most relevant languages including object-oriented features. Objective: This paper provides a complete evaluation of class operators for the C++ programming language. MuCPP, a new system devoted to the application of mutation testing to this language, was developed to this end. This mutation system implements class mutation operators in a robust way, dealing with the inherent complexity of the language. Method: MuCPP generates the mutants by traversing the abstract syntax tree of each translation unit with the Clang API, and stores mutants as branches in the Git version control system. The tool is able to detect duplicate mutants, avoid system headers, and drive the compilation process. Then, MuCPP is used to conduct experiments with several open-source C++ programs. Results: The improvement rules listed in this paper to reduce unproductive class mutants have a significant impact in the computational cost of the technique. We also calculate the quantity and distribution of mutants generated with class operators, which generate far fewer mutants than their traditional counterparts. Conclusions: We show that the tests accompanying these programs cannot detect faults related to particular object-oriented features of C++. In order to increase the mutation score, we create new test scenarios to kill the surviving class mutants for all the applications. The results confirm that, while traditional mutation operators are still needed, class operators can complement them and help testers further improve the test suite.",
keywords = "C++, Class mutation operators, Mutation system, Mutation testing, Object-oriented programming",
author = "Pedro Delgado-P{\'e}rez and Inmaculada Medina-Bulo and Francisco Palomo-Lozano and Antonio Garc{\'i}a-Dom{\'i}nguez and Dom{\'i}nguez-Jim{\'e}nez, {Juan Jos{\'e}}",
note = "{\circledC} 2016, Elsevier. Licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International http://creativecommons.org/licenses/by-nc-nd/4.0/ Corrigendum: Delgado-P{\'e}rez, P., Medina-Bulo, I., Palomo-Lozano, F., Garc{\'i}a-Dom{\'i}nguez, A., & Dom{\'i}nguez-Jim{\'e}nez, J. J. (2016). Corrigendum to 'Assessment of class mutation operators for C++ with the MuCPP mutation system' [Information and Software Technology, 81, (2017) 169-184]. Information and Software Technology. DOI: 10.1016/j.infsof.2017.02.001",
year = "2017",
month = "1",
day = "1",
doi = "10.1016/j.infsof.2016.07.002",
language = "English",
volume = "81",
pages = "169--184",
journal = "Information and Software Technology",
issn = "0950-5849",
publisher = "Elsevier",

}

Assessment of class mutation operators for C++ with the MuCPP mutation system. / Delgado-Pérez, Pedro; Medina-Bulo, Inmaculada; Palomo-Lozano, Francisco; García-Domínguez, Antonio; Domínguez-Jiménez, Juan José.

In: Information and Software Technology, Vol. 81, 01.01.2017, p. 169-184.

Research output: Contribution to journalArticle

TY - JOUR

T1 - Assessment of class mutation operators for C++ with the MuCPP mutation system

AU - Delgado-Pérez, Pedro

AU - Medina-Bulo, Inmaculada

AU - Palomo-Lozano, Francisco

AU - García-Domínguez, Antonio

AU - Domínguez-Jiménez, Juan José

N1 - © 2016, Elsevier. Licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International http://creativecommons.org/licenses/by-nc-nd/4.0/ Corrigendum: Delgado-Pérez, P., Medina-Bulo, I., Palomo-Lozano, F., García-Domínguez, A., & Domínguez-Jiménez, J. J. (2016). Corrigendum to 'Assessment of class mutation operators for C++ with the MuCPP mutation system' [Information and Software Technology, 81, (2017) 169-184]. Information and Software Technology. DOI: 10.1016/j.infsof.2017.02.001

PY - 2017/1/1

Y1 - 2017/1/1

N2 - Context: Mutation testing has been mainly analyzed regarding traditional mutation operators involving structured programming constructs common in mainstream languages, but mutations at the class level have not been assessed to the same extent. This fact is noteworthy in the case of C++, despite being one of the most relevant languages including object-oriented features. Objective: This paper provides a complete evaluation of class operators for the C++ programming language. MuCPP, a new system devoted to the application of mutation testing to this language, was developed to this end. This mutation system implements class mutation operators in a robust way, dealing with the inherent complexity of the language. Method: MuCPP generates the mutants by traversing the abstract syntax tree of each translation unit with the Clang API, and stores mutants as branches in the Git version control system. The tool is able to detect duplicate mutants, avoid system headers, and drive the compilation process. Then, MuCPP is used to conduct experiments with several open-source C++ programs. Results: The improvement rules listed in this paper to reduce unproductive class mutants have a significant impact in the computational cost of the technique. We also calculate the quantity and distribution of mutants generated with class operators, which generate far fewer mutants than their traditional counterparts. Conclusions: We show that the tests accompanying these programs cannot detect faults related to particular object-oriented features of C++. In order to increase the mutation score, we create new test scenarios to kill the surviving class mutants for all the applications. The results confirm that, while traditional mutation operators are still needed, class operators can complement them and help testers further improve the test suite.

AB - Context: Mutation testing has been mainly analyzed regarding traditional mutation operators involving structured programming constructs common in mainstream languages, but mutations at the class level have not been assessed to the same extent. This fact is noteworthy in the case of C++, despite being one of the most relevant languages including object-oriented features. Objective: This paper provides a complete evaluation of class operators for the C++ programming language. MuCPP, a new system devoted to the application of mutation testing to this language, was developed to this end. This mutation system implements class mutation operators in a robust way, dealing with the inherent complexity of the language. Method: MuCPP generates the mutants by traversing the abstract syntax tree of each translation unit with the Clang API, and stores mutants as branches in the Git version control system. The tool is able to detect duplicate mutants, avoid system headers, and drive the compilation process. Then, MuCPP is used to conduct experiments with several open-source C++ programs. Results: The improvement rules listed in this paper to reduce unproductive class mutants have a significant impact in the computational cost of the technique. We also calculate the quantity and distribution of mutants generated with class operators, which generate far fewer mutants than their traditional counterparts. Conclusions: We show that the tests accompanying these programs cannot detect faults related to particular object-oriented features of C++. In order to increase the mutation score, we create new test scenarios to kill the surviving class mutants for all the applications. The results confirm that, while traditional mutation operators are still needed, class operators can complement them and help testers further improve the test suite.

KW - C++

KW - Class mutation operators

KW - Mutation system

KW - Mutation testing

KW - Object-oriented programming

UR - http://www.scopus.com/inward/record.url?scp=84994257635&partnerID=8YFLogxK

U2 - 10.1016/j.infsof.2016.07.002

DO - 10.1016/j.infsof.2016.07.002

M3 - Article

AN - SCOPUS:84994257635

VL - 81

SP - 169

EP - 184

JO - Information and Software Technology

JF - Information and Software Technology

SN - 0950-5849

ER -