An empirical comparison between direct and indirect test result checking approaches

Abstract:An oracle in software testing is a mechanism for checking whether the system under test has behaved correctly for any executions. In some situations, oracles are unavailable or too expensive to apply. This is known as the oracle problem. It is crucial to develop techniques to address it, and metamorphic testing (MT) was one of such proposals. This paper conducts a controlled experiment to investigate the cost effectiveness of using MT by 38 testers on three open-source programs. The fault detection capability and time cost of MT are compared with the popular assertion checking method. Our results show that MT is cost-efficient and has potentials for detecting more faults than the assertion checking method.
Grants:HKU 7145/04E
Citation:Peifeng Hu, Zhenyu Zhang, W.K. Chan and T.H. Tse, "An empirical comparison between direct and indirect test result checking approaches," in Proceedings of the Third International Workshop on Software Quality Assurance (SOQUA 2006) , in conjunction with the 14th ACM SIGSOFT Symposium on Foundations of Software Engineering (SIGSOFT 2006/FSE-14), pages 6-13, ACM Press, New York, USA (2006).
Remarks:[Acceptance Rate: 41.6%, 10 out of 24]
Related Papers:-

Selected Tags

Tag Groups


ACM SigSoft
IEEE Software Engineering Online