hadoop 2.4.1 部署文档.docx

环境软件环境Centos6.5 x64jdk-8u5-linux-x64.gzhadoop-2.4.1.tar.gz （官方提供的是32bit的，64bit的需要自己下载源码编译）zookeeper-3.4.6.tar.gzHostsvm01vm02vm03vm05vm04主机角色Vm01Vm02Vm03Vm04Vm05NameNodeYesYesNoNoNoDataNodeNoNoYesYesYesJournalNodeNoNoYesYesYesZkfcServer(QuorumPeerMain)YesYesYesYesYesDFSZKFailoverControllerYesYesNoNoNo部署配置ssh无密码验证新建用户及设置密码#useradd hadoop echo hadoop |passwd --stdin hadoop创建一个无密码秘钥$ ssh-keygen (一直默认回车)拷贝公钥文件到其他机器$ssh-copy-id -i ~/.ssh/id_rsa.pub vm02$ssh-copy-id -i ~/.ssh/id_rsa.pub vm03$ssh-copy-id -i ~/.ssh/id_rsa.pub vm04$ssh-copy-id -i ~/.ssh/id_rsa.pub vm05连接测试$ssh vm02$ssh vm03$ssh vm04$ssh vm05安装jdk#tar xf jdk-8u5-linux-x64.gz#mv jdk1.8.0_05/ /usr/local/#java –version注：偷个懒，我把环境变量配置文件先全部贴在这里了#cat/etc/profileexport JAVA_HOME=/usr/local/jdk1.8.0_05export HADOOP_HOME=/home/hadoop/hadoop-2.4.1/export ZOOKEEPER_HOME=/home/hadoop/zookeeper-3.4.6export CLASSPATH=.:$JAVA_HOME/lib/tools.jar:$JAVA_HOME/lib/dt.jarexport PATH=$PATH:$JAVA_HOME/bin:$HADOOP_HOME/sbin/:$HADOOP_HOME/bin/:$ZOOKEEPER_HOME/bin部署zookeeper集群$tar xf zookeeper-3.4.6.tar.gz$cd zookeeper-3.4.6/conf/$cp zoo_sample.cfg zoo.cfg有效配置文件$egrep -v #|^$ zoo.cfg tickTime=2000initLimit=10syncLimit=5dataDir=/home/hadoop/zookeeper-3.4.6/dataclientPort=2181server.1=vm01:2888:3888server.2=vm02:2888:3888server.3=vm03:2888:3888server.4=vm04:2888:3888server.5=vm05:2888:3888新建zookeeper data目录$mkdir /home/hadoop/zookeeper-3.4.6/data$cd /home/hadoop/zookeeper-3.4.6/data写入myid$echo 1 myid注：此处的1和server.1对应，vm01中的myid写为1，vm02中的myid写为2，copy到其他机器后依次修改……hadoop部署安装$tar xf hadoop-2.4.1_x64.tar.gzhadoop-env.sh$cat hadoop-env.sh配置java路径export JAVA_HOME=/usr/local/jdk1.8.0_05slaves$cat slaves vm03vm04vm05core-site.xml$vim core-site.xmlconfigurationproperty namefs.defaultFS/name valuehdfs://mycluster/value/propertyproperty nameha.zookeeper.quorum/name valuevm01:2181,vm02:2181,vm03:2181,vm04:2181,vm05:2181/value/property/configurationhdfs-site.xml $vim hdfs-site.xmlconfigurationpropertynameservices/namevaluemycluster/value/propertyp