yarn 工作流程.pptxVIP

yarn那点事 讲说人:朱钰锋 Client ApplicationManager Resource Scheduler NodeManager NodeManager Map Task MR App Mstr Container Container ReduceTask Map Task Container Container ResourceManager 1 1.client向yarn提交job，首先找ResourceManager 分配资源 Client ApplicationManager Resource Scheduler NodeManager NodeManager Map Task MR App Mstr Container Container ReduceTask Map Task Container Container ResourceManager 2 2.ResourceManager开启一个Container,在Container中运行一个Application manager 1 Client ApplicationManager Resource Scheduler NodeManager NodeManager Map Task MR App Mstr Container Container ReduceTask Map Task Container Container ResourceManager 3 3.Application manager找一台nodemanager启动Application master，计算任务所需的计算 1 2 Client ApplicationManager Resource Scheduler NodeManager NodeManager Map Task MR App Mstr Container Container ReduceTask Map Task Container Container ResourceManager 4 4.Application master向Application manager（Yarn）申请运行任务所需的资源 1 2 3 Client ApplicationManager Resource Scheduler NodeManager NodeManager Map Task MR App Mstr Container Container ReduceTask Map Task Container Container ResourceManager 5 5.Resource scheduler将资源封装发给Application master 1 4 2 3 Client ApplicationManager Resource Scheduler NodeManager NodeManager Map Task MR App Mstr Container Container ReduceTask Map Task Container Container ResourceManager 6.Application master将获取到的资源分配给各个nodemanager 6 6 1 4 2 3 5 Client ApplicationManager Resource Scheduler NodeManager NodeManager Map Task MR App Mstr Container Container ReduceTask Map Task Container Container ResourceManager 7 7.各个nodemanager得到任务和资源开始执行map task 7 1 3 6 4 2 5 6 Client ApplicationManager Resource Scheduler NodeManager NodeManager Map Task MR App Mstr Container Container ReduceTask Map Task Container Container ResourceManager 8 8.map task执行结束后，开始执行reduce task 8 1 3 6 4 7 5 7 6 Client ApplicationManager Resource Scheduler NodeManager NodeManager Map Task MR App Mstr Container Container ReduceTask Map Task Container Container ResourceManager 8 9.map task和 redu