ITier开发优化实践.pptx

ITier开发优化实践爱多@CN凤吟@T/windyrobin孤独/windyrobin的登山/windyrobin人/windyrobin@/ 张严明(Edward zhang）Agenda ITier架构 基本策略 优化历程 优化总结 思维规范ITier 位置数据源主站备库RAC主站日志DataX / TimeTunnel计算层Hadoop集群实时流数据存储层MyFOXProm查询层数据中间层 / ITier数据魔方淘宝指数开放API产品ITier 架构基本策略 Http : keep-aliveNode V0.5.3 +var agent = new http.Agent({ maxSockets : 100});var options = { agent : agent …}http.get(options. Function(res){ …});基本策略 Config : auto-reloadvar self = this;…fs.watchFile(fpath ,function(curr, prev){ if(self.load_timer) return; if(curr.ctime.toString() != prev.ctime.toString()){ self.load_timer = setTimeout(function(){ self.loadCfg(fpath); },LOAD_DELAY); }});基本策略 Socket Pool – Generic Poolvar poolModule = require(generic-pool);var pool = poolModule.Pool({ name : mysql, create : function(callback){ var Client = require(mysql).Client; var c = new Client(); … }, destroy : function(client){ client.end(); }, max : Cfg.maxSockets, idleTimeoutMillis : Cfg.timeout});基本策略 ClusterMulti-Worker-Listen Master.send(listenfd) Worker.listen(…) Master.close(listenfd);Master-Listen (we use this for fully control) Master.listen() Master.send(sockfd) Worker.receive(sockfd) CreateReqRes(sockfd)优化历程First TimeSiegeKeep-aliveSingel core ‘hello,world’ : 8000qps(5 cpu core) 4 workers260 QPS !!! WHY ?? 优化历程Thinking…优化历程Thinking…Calculate Each-step OverheadHttp Requset(skip other steps:) Unit/Benchmark Testvar TEST_NUMBER = 100*1000;var start = new Date();for(var i=0; i TEST_NUMBER; i++){ step1();}var end = new Date();console.log(‘time : ’ + (end – start));优化历程Step-overhead …URL - SQL conversion : 0.8msMain control : 0.5msTable LeftJoin (2000 X 2000) : 13msEquation: 1S / (13 + 0.8 + 0.5)ms * 4 == 280优化历程OptimizationURL - SQL conversion Pre-parsing / Template keywords replacement var obj = { … id : ‘#id’, thedate : ‘#date’}Result:0.8ms ? 0.05 ms 优化历程OptimizationTable LeftJoin (2000 X 2000) Language Skills 避免重复计算 Array / Object 遍历效率斟酌 Array.push / array[i] 比较 引入临时量，避免对象多次查找 String ‘+’ 与 array.join(‘