An experimental study to compare the use of metamorphic testing and assertion checking to alleviate the test oracle problem

Abstract:A test oracle in software testing is a mechanism for checking whether the program under test behaves correctly for any execution. In some practical situations, oracles can be unavailable or too expensive to apply. Metamorphic testing (MT) was proposed to alleviate this problem so that software can be delivered under the time-to-market pressure. However, the effectiveness of MT has not been studied adequately. This paper conducts a controlled experiment to investigate the cost effectiveness of using MT. The fault detection capability and time cost of MT are compared with the standard assertion checking method. Our results show that MT has potentials to detect more faults than the assertion checking method. The experiment results also show a trade-off between the two testing methods: MT can be less efficient but more effective, and can be defined at a coarser level of granularity than the assertion checking method.
Grants:GRF 111107, GRF 717308, ARC DP0984760
Links:PDF DOI
Citation:Zhenyu Zhang, W.K. Chan, T.H. Tse, Peifeng Hu, "An experimental study to compare the use of metamorphic testing and assertion checking to alleviate the test oracle problem", to appear in Journal of Software, 20(10):2637-2654, Institute of Software, Chinese Academy of Sciences, 2009.
 



张震宇,陈荣光,谢俊谦,胡佩锋.蜕变测试和断言检查的比较与实验研究.软件学报,2009,20(10):2637-2654



Remarks:[15 Jan 2009] 
Related Papers:-

Selected Tags

Tag Groups

Links

ACM SigSoft
IEEE Software Engineering Online