数据挖掘取样方法研究 - 科学网.pdf

计算机研 究与发展 ISSN 1000—12391CN 11-1777门rP JournaIofComputerResearchandDevelopment 48(1)：45—54，2O11 数据挖掘取样方法研究 胡文瑜 孙志挥 吴英杰 。 (东南大学计算机科学与工程学 院 南京 210096) (福建工程学院计算机与信息科学系 福州 350108) 。(福州大学数学与计算机科学学 院 福州 350108) (huwenyu@fjut．edu．cn) StudyofSamplingM ethodsonDataM iningandStream M ining HuWenyu 。SunZhihui。WuYingjie’。 (SchoolofComputerScienceandEngineering，SoutheastUniversity，Nanjing210096) (DepartmentofComputerandInformationScience，F JanUniversityofTechnology，Fuzhou350108) 。(CollegeofMathematicsandComputerScience，FuzhouUniversity，Fuzhou350108) Abstract Samplingisanefficientandmostwidely—usedapproximation technique．Itenableslotsof algorithmstobeappliedtohugedatasetbyuseofscalingdowndramaticallydatasetfordatamining and streaming mining． Throughoutthe detailed review ，akind of taxonomic frame ofsampling algorithmsbased on uniform sampling and biased sampling is presented；meanwhile， analysis， comparisonsandevaluationsofrepresentativesamplingalgorithmssuchasreservoirsampling，concise sampling，countsampling，chain—sampling，DV sampling and SO on are performed． Due to the limitationsofuniform sampling insomeapplications-- querieswith relatively low selectivity，outlier detection in largemultidimensionaldata sets，and clustering over data streamswith skewed Zipf distribution，the importanceofneed for using biased sampling methods in thesescenarios isfully dissertated．In addition to listing successfulapplicationsof sampling techniques in data mining， statisticsestimatingand stream mining up tonow，we survey theapplication and developmentof sampling techniques，especially those traditionalclassic sampling techniques such as progressive sampling，adaptive sampling， stratified sampling and two—phase sampling etc． Finally， future challengesanddirectionswithrespecttodatastream samplingarefurtherdiscussed． Keywords datamining；uniform