A Neural Network Classifier for Junk E-Mail.ppt

A Neural Network Classifier for Junk E-Mail Ian Stuart, Sung-Hyuk Cha, and Charles Tappert CSIS Student/Faculty Research Day May 7, 2004 Spam, spam, spam, … Fighting spam Several commercial applications exist Server-side: expensive Client-side: time-consuming No approach is 100% effective Spammers are aggressive and adaptable Best solutions are typically hybrids of different approaches and criteria Common approaches Simple filters Common words or phrases Unusual punctuation or capitalization Blacklisting: “just say NO” (if you can) Reject e-mail from known spammers Whitelisting: “friends only, please” Accept e-mail only from known correspondents Classifiers: examine each e-mail and decide Only a few publications on spam classifiers Na?ve Bayesian classifiers Used in commercial classifiers Assumes recognition features are independent Max likelihood = product of likelihoods of features E-mail classifier – examines each word Training assigns a probability to each word Look up each word/probability in a dictionary If the product of the probabilities exceeds a given threshold, it is spam Challenge – creating the “dictionary” We compare our Neural Network against two published Na?ve Bayesian classifiers Na?ve Bayesian classifier issues How many features (words), which ones? How is degradation avoided as spammers’ vocabulary changes? What values are assigned to new words? What are the thresholds? How to avoid “sabotage” of classifier? Which one isn’t spam?(subject headers) 5 Be a mighty warrior in bed! vcrhwt ygjztyjjh Money Back Guarantee_HGH kindle life pddez liw mzac v a l i u m - D i a z e p a m used to relieve anxiety Fairfield tennis schedule :Dramatic E,nhancement fo=r .Men = ffumqid ,Refinance now. Dont wait Which one isn’t spam? (subject headers) 5 Be a mighty warrior in bed! vcrhwt ygjztyjjh Money Back Guarantee_HGH kindle life pddez liw mzac v a l i u m - D i a z e p a m used to relieve anxiety Fairfield tennis schedule :Dramatic E,nhancement fo=r .Men = f