云储存海量数据采集方法研究.docVIP

云储存海量数据的采集方法研究 　　摘 要： 针对传统数据采集方法主要通过数据特征采集，忽略采集过程对数据特征造成的影响，导致数据采集耗时长、误差大的问题，提出基于REID技术与F统计计量结合的云储存海量数据采集方法。在分析数据采集原理的基础上，对云储存的原始数据进行非线性补偿，设置参数对数据进行预处理，建立一种能够对内存进行直接访问的硬件机制，给出部分传输程序；采用聚类算法对云储存数据进行聚类，结合F统计计量进行检验所建立的判别函数的有效性，实现对云储存海量数据的采集。实验结果表明，采用改进方法进行云储存数据采集时，其采集结果相比传统方法精度及完整度均有提高，具有一定的优势。 　　关键词： 云储存； 海量数据采集； REID技术； F统计计量； 非线性补偿； 聚类算法 　　中图分类号： TN911?34； TP391 文献标识码： A 文章编号： 1004?373X（2018）14?0010?04 　　Research on acquisition method of cloud storage mass data 　　DENG Wenwen1，2， SUN Chengming3， QIN Peiliang2 　　（1. School of Accounting Information Systems， Virginia Polytechnic Institute and State University， Virginia 24061， U.S.A； 　　2. Smart Agriculture School of Suzhou Polytechnic Institute of Agriculture， Suzhou 215008， China； 　　3. Agricultural College of Yangzhou University， Yangzhou 225127， China） 　　Abstract： In allusion to the problems of long time consumption and big error of data acquisition existing in the traditional data collection method， in which the influence of acquisition process on data features is ignored due to its data feature acquisition， a cloud storage mass data acquisition method based on the combination of REID technology and F statistical metrology is proposed. On the basis of analyzing the data acquisition principle， nonlinear compensation is conducted for the cloud storage original data， some parameters are set for data preprocessing to establish a hardware mechanism that can directly access the memory， and part of transmission programs are given. The clustering algorithm is adopted to cluster the cloud storage data， and the F statistical metrology is combined to detect the effectiveness of the established discrimination function， so as to realize the acquisition of cloud storage mass data. The experimental results show that the acquisition precision and integrity of the improved cloud storage data acquisition method are higher than those of the traditional data acquisition method. 　　Keywords： cloud storage； mass data acquisition