Testing object-oriented industrial software without precise oracles or results

Abstract:From 30% to 50% of the resources in an average software project are typically allotted to testing. Still, inadequate software testing costs industry $22 billion to $60 billion per year in the U.S. alone [8]. We would all spend less if software engineers could use more effective testing methods and automated testing tools. On the other hand, testing is very difficult in real-world projects. Software testing is commonly accomplished by defining the test objectives, selecting and executing test cases, and checking results [2]. Although many studies have concentrated on the selection of test cases, checking test results is not trivial. Two problems are often encountered: How to determine success or failure. A test oracle [11] is the mechanism for specifying the expected outcome of software under test, allowing testers to check whether the actual result has succeeded or failed. 1 In theory, test oracles can be determined by the software specification. In practice, however, a specification may provide only high-level descriptions of the system and cannot possibly include all implementation details. Hence, software testers must also rely on domain knowledge and user judgment to evaluate results. Such manual efforts are often error prone [9].
Grants:ASM UIM/77, GRF 714504, CityU 7200079
Links:PDF, DOI
Citation:T.H. Tse, Francis C.M. Lau, W.K. Chan, Peter C. K. Liu, and Colin K. F. Luk, "Testing object-oriented industrial software without precise oracles or results", Communications of the ACM  50(8):78 - 85 (2007)
Remarks:[Impact factor 2007: 1.593]
Related Papers:-

Selected Tags

Tag Groups

Links

ACM SigSoft
IEEE Software Engineering Online