Analysis of heuristic rule evaluation measures.pdfVIP

Filip Železný Nada Lavra Department of Cybernetics Josef Stefan Institute Czech Technical University Jamova 39 Prague 10000 Ljublj ana Czech Republic Slovenia Zelezny@labe.felk.cvut.cz Nada.Lavrac@ij s.si Analysis of Heuristic Rule Evaluation Measures J. Stefan Institute Technical Report IJS – DP – 8042, March 1999 Abstract Numerous methods are used f or p erf ormance evaluation in machine learning and knowledge discovery . In inductive logic p rogramming (ILP) - an imp ortant subf ield of KDD - where the goal is the automatic construction of hyp othesis (def ined in af ormal language) on the basis of an extensional set of f acts (examp les) and certain background knowledge, various measures are used to select the most p romising element (clause) to be p art of the current hyp othesis. In the p resence of noise (errors) in the inp ut data, we usually trade off between the clause ’s accuracy and generality . The trade-off is exp ressed by a clause-quality measure (heuristic) . Recently, some new heuristics have been introduced [2] . In this p ap er we comp are the new and some classical heuristics in both analy tical and exp erimental way . We also f ormally introduce the inf luence of noise in data and survey how if aff ects the heuristics. I. Introduction Numerous measures are used for performance evaluation in machine learning and knowledge discovery. In classification-oriented p redictive induction, the most frequently used measure is class