处理海量数据78924（Deal with mass data 78924）.docVIP

处理海量数据78924（Deal with mass data 78924） Deal with mass data,.Txt, dont give up on yourself! (mom once said to me, the moment I turned round to go out, I burst into tears and didnt want anyone to see it!) To see this one small heart feeling, think of when I left home to thousands of miles away, my mother also said something similar, but as a man, you must create a day, in order to repay their parents!! In the actual work, I am lucky enough to have access to massive data processing problems, and dealing with them is an arduous and complex task. The reasons are as follows: First, the amount of data is too large, the data may exist in any case. If there are 10 data, so can each to check for handling, if there are hundreds of data, can also be considered, if the data to the level of tens of millions, even billions of dollars, that is not the hand can solve, must be processed by tools or programs, especially the mass data, what are there may be, for example, data somewhere in the format of a problem, especially in the treatment program, but also in front of the normal processing, suddenly to a place there, to terminate the program. Two, software and hardware need high, system resource occupation rate is high. Dealing with massive data, in addition to good methods, the most important thing is to use tools reasonably and allocate the system resources reasonably. In general, if the data over TB, minicomputer is to be considered, the ordinary machine if there is a good way to consider, but also must increase CPU and memory, like to have the courage not light thousands upon thousands of horses and soldiers, a soldier is difficult to win. Three, the need for high processing methods and skills. This is also the purpose of writing this paper. The good method is the accumulation of long-term work experience of an engineer, and also a summary of personal experience. There are no general methods of handling, but there are general principles and rules. Well, what are the experience an