AnInvitationtoData-Mining-RomanpoetHome.ppt

An Invitation to Data-Mining Virgil -- virgil@ GregR -- gregr@ Interz0ne IV March 12, 2005 Lecture Outline Introducing Data-Mining Google Hacking Intermission Examples of Using Data-Mining for: Money Power Sex Closing The Advent of Databases and the Internet Fact: The amount of data we have access to is greater than ever before and is still growing exponentially. If nothing else, the continued archival of current data will quickly add up. Continued Growth of the Internet Growth of Digital Information A Practical Example… Back in the old days news of interesting websites propagated through word of mouth. Then it moved to USENET groups (blogs are a modern equivalent). But, then it became difficult to find the hottest newsgroups. To compensate for this we started using search engines. Today, we’re frequently using meta-search engines meta-blogging sites like , , and del.icio.us. Data-Mining is an increasingly a powerful tool to take advantage of the availability of huge amounts of digitized information. What is Data-Mining… From Wikipedia...Data mining is been defined as [1] The nontrivial extraction of implicit, previously unknown, and potentially useful information from data” [2] The science of extracting useful information from large data sets or databases. Like Artificial Intelligence, “Data-Mining” is an widely used term with general connotations. What is Data-Mining (contd.) Data-Mining is usually broken up into two distinct steps. 1. “Data-Warehousing” – Collecting large amounts of data 2. “Mining / Extraction” – Analysis (often statistical) of the collected information: Some Examples of Data Mining... A’s Recommendation System MusicP National Security Agency’s ECHELONECHELON is the largest electronic spy network in history, run by the United States, the United Kingdom, Canada,  Australia, and New Zealand. It captures telephone calls,  faxes, e-mails, and IMs from around the world. ECHELON is estimated to intercept about 3 billion commun