antnest：一种支持多计算模型的分布式计算系统word格式论文.docxVIP

AbstractWith the development of computer and the Internet, the growth of the information has reached an unprecedented speed, the increasing accumulation of massive data makes people ushered in the big data times. How to extract the useful information from these huge data correctly and efficiently and make a proper decision becomes a challenging task. Facing this problem, people tend to consider using a distributed computing for processing. However, for most of the information enterprise, always have many business with different needs, they applicable to different distributed computing platform, which led to a wide variety of distributed computing platform, to learn and maintain these platforms often requires a very high cost, this bring huge economic pressure to new Internet enterprise. Therefore, Design a unified distributed computing platform becomes the urgent needs of each enterprise.Antnest catagorizes three computation models for the enterprise data processing, and gives a comparison for them. We propose a computation model to unify the three computat ion models after founding they are equivalent. The proposed computation model mainly e mploys a same data source interface and data processing interface. Antnest composite the characteristics of batch distributed computing, streaming distributed computing and automaticschedulingtaskframework,providegoodsupporttothreebasic enterprise business——Large-scale static dataprocessing business, online real-time business,background timingbusiness,toreducethe enterprise overhead causedby the useandmaintenance of avarietyofplatforms. Antnest provides users with a unified task create mode, users can use the same rules to define data sources and calculation unit configuration file according to their actual business needs, and processing logic calculation unit in the corresponding task interface and then be able to upload tasks to running in the platform. Antnest shields the difference betweenthetas