Lecture Notes 16 Bayes Theorem and Data Mining.ppt

Lecture Notes 16: Bayes’ Theorem and Data Mining Zhangxi Lin ISQS 6347 Modeling Uncertainty Probability Review Bayes Classifier Value of Information Conditional Probability and Bayes’ Theorem Expected Value of Perfect Information Expected Value of Imperfect Information Probability Review P(A|B) = P(A and B) / P(B) “Probability of A given B” Example, there are 40 female students in a class of 100. 10 of them are from some foreign countries. 20 male students are also foreign students. Even A: student from a foreign country Even B: a female student If randomly choosing a female student to present in the class, the probability she is a foreign student: P(A|B) = 10 / 40 = 0.25, or P(A|B) = P (A B) / P (B) = (10 /100) / (40 / 100) = 0.1 / 0.4 = 0.25 That is, P(A|B) = # of AB / # of B = (# of AB / Total) / (# of B / Total) = P(A B) / P(B) Venn Diagrams Probability Review Complement Bayes Classifier Bayes’ Theorem (From Wikipedia) In probability theory, Bayes theorem (often called Bayes Law) relates the conditional and marginal probabilities of two random events. It is often used to compute posterior probabilities given observations. For example, a patient may be observed to have certain symptoms. Bayes theorem can be used to compute the probability that a proposed diagnosis is correct, given that observation. As a formal theorem, Bayes theorem is valid in all interpretations of probability. However, it plays a central role in the debate around the foundations of statistics: frequentist and Bayesian interpretations disagree about the ways in which probabilities should be assigned in applications. Frequentists assign probabilities to random events according to their frequencies of occurrence or to subsets of populations as proportions of the whole, while Bayesians describe probabilities in terms of beliefs and degrees of uncertainty. The articles on Bayesian probability and frequentist probability discuss these debates at greater length. Bayes’ Theorem Example of Bay