Evaluating Testing Methods by Delivered Reliability.ppt

Empirical Tool – version 1 5K repetitions of single test Relation of results to Frankl Evaluating Testing Methods by Delivered Reliability Frankl, Hamlet, Littlewood, Strigini IEEE TOSE Aug98 Multiple Failure,debug w/o sub E(Q) = Sqi(1-di)T E(Q) = Sqi(1-qi)T Example – operational 5 tests Flt 1 Flt 2 Flt 3 Flt 4 Flt 5 SD 1 10/24 0/24 0/24 0/24 0/24 SD 2 0/14 5/14 0/14 0/14 0/14 SD 3 0/14 5/14 1/14 1/14 0/14 SD 4 0/24 0/24 2/24 0/24 0/24 SD 5 0/24 0/24 2/24 0/24 1/24 Example 3 – Solve empirically Flt 1 Flt 2 Flt 3 Flt 4 Flt 5 SD 1 10/20 0/20 0/20 0/20 0/20 SD 2 0/10 5/10 0/10 0/10 0/10 SD 3 0/1