hadoop大数据用开发环境配置.docxVIP

Hadoop大数据应用开发环境配置运行环境计算机名IP角色备注Kingo-mn30主节点(namenode), masters负责总管分布式数据的调度。Kingo-ms31副节点(JobTracker), masters分解任务的执行。资源控制器Kingo-d132数据节点1(Datanode1)Slaves/TaskTracker，负责分布式数据存储以及任务的执行。Kingo-d238数据节点2(Datanode2) Kingo-d339数据节点3(Datanode3) 说明修改 /etc/sysconfig/network文件可以修改主机名；主机名不能含“_”。Ip主机名程序进程30Kingo-mnJdkHadoophbaseNamenodeDFSZKFailoverControllerHamsterresourceManager31Kingo-msJdkHadoophbaseNamenodeSecondaryNamenodeJobTrackerDFSZKFailoverControllerHamsterresourceManager32Kingo-d1JdkHadoopZookeeperhbaseDatanodenodeManagerJournalNodeQuorumPeerMainHregionServerTaskTracker38Kingo-d2JdkHadoopZooKeeper hbaseDatanodenodeManagerJournalNodeQuorumPeerMainHregionServerTaskTracker39Kingo-d3JdkHadoopZookeeperHbaseDatanodenodeManagerJournalNodeQuorumPeerMainHregionServerTaskTracker创建帐号四台机器均创建kingo用户组和bigdata帐号，密码统一为Kingo123。#groupadd kingo# useradd -d /home/bigdata -g kingo bigdata#passwd bigdata //设置密码为Kingo123#cd /home #chgrp -R kingo bigdata //设置目录所属组#chmod -R g=rwx bigdata //设置目录所属组的权限为读、写、执行绑定hostname与IP#vi /etc/hosts，增加内容如下：30???????? kingo-mn31???????? kingo-ms38???????? kingo-rm32???????? kingo-d139???????? kingo-d2设置节点之间的免密码通信向各节点复制Hosts文件scp /etc/hosts kingo-ms:/etc/hosts?scp /etc/hosts kingo-d1:/etc/hostsscp /etc/hosts kingo-d2:/etc/hosts?scp /etc/hosts kingo-d3:/etc/hosts?在各节点上生成各自SSH秘钥对#ssh-keygen -t dsa -P -f ~/.ssh/id_dsa#cd ~/.ssh#cat id_dsa.pubauthorized_keys#chmod 600 authorized_keys3、复制公钥到某一节点#ssh-copy-id -i ~/.ssh/id_dsa.pub kingo-mn4、复制密钥到各个节点。在kingo-mn上执行如下命令：#scp /root/.ssh/authorized_keys kingo-ms:/root/.ssh/;scp /root/.ssh/authorized_keys kingo-d1:/root/.ssh/;scp /root/.ssh/authorized_keys kingo-d2:/root/.ssh/; scp /root/.ssh/authorized_keys kingo-d3:/root/.ssh/安装JDK把jdk-7u80-linux-x64.rpm安装包复制到/bigdata/setup文件夹下，执行：#rpm –ivh jdk-7u80-linux-x64.rpmJdk安装到默认目录/usr/java/jdk1.7.0_80下。#vi /etc/profile加入如下配置信息：export JAVA_HOME=/usr/java/jdk1.7.0_80export CLASSPATH=.:$JAVA_HOME/lib/tools.jar:$JAVA_HOME/lib/dt.jarexport PATH=.:$JAVA_HOME/bin:$PATH#source pro