虚拟机中centos6.6hadoop环境搭建.docx

一、Hadoop安装1、解压文件后的目录配置之前，先在本地文件系统创建以下文件夹：mkdir -p /home/hadoop/tmp/home/dfs/data/home/dfs/name4、进去hadoop配置文件目录[root@master hadoop-2.6.0]# cd etc/hadoop/4.1、配置 hadoop-env.sh文件--修改JAVA_HOME# The java implementation to use.export JAVA_HOME=/opt/jdk1.8.0_664.2、配置 yarn-env.sh 文件--修改JAVA_HOME# some Java parametersexport JAVA_HOME=/opt/jdk1.8.0_664.3、配置slaves文件--增加slave节点?slave1slave24.4、配置 core-site.xml文件--增加hadoop核心配置（hdfs文件端口是9000、file:/home/spark/opt/hadoop-2.6.0/tmp、）configurationproperty?namefs.defaultFS/name??valuehdfs://master.hadoop:9000/value/propertyproperty??nameio.file.buffer.size/name??value131072/value/propertyproperty??namehadoop.tmp.dir/name??valuefile:/home/hadoop/tmp/value??descriptionAbasefor other temporary directories./description/propertyproperty??namexyuser.root.hosts/name??value*/value/propertyproperty??namexyuser.root.groups/name??value*/value/property!--指定zookeeper地址--?property??????nameha.zookeeper.quorum/name??????valuemast1:2181,mast2:2181,mast3:2181/value?/property/configuration4.5、配置??hdfs-site.xml 文件--增加hdfs配置信息（namenode、datanode端口和目录位置）configurationproperty??namenode.secondary.http-address/name??valuemaster.hadoop:9001/value/property??property? ?name.dir/name? ?valuefile:/home/dfs/name/value/propertyproperty??namedfs.datanode.data.dir/name??valuefile:/home/dfs/data/value??/propertyproperty??namedfs.replication/name??value3/value/propertyproperty??namedfs.webhdfs.enabled/name??valuetrue/value/property?!-- 开启NameNode故障时自动切换 --????property??????????namedfs.ha.automatic-failover.enabled/name??????????valuetrue/value????/property????!-- 配置失败自动切换实现方式 --????property????????????namevider.ns/name????????????valuenode.ha.ConfiguredFailoverProxyProvider/value????/property/configuration4.6、配置??mapred-site.xml 文件--增加mapreduce配置（使用yarn框架、jobhistory使用地址以及web地址）configuration??property? ?name/name? ?valueyarn/value/propertyproperty??namemapreduce.jobhistory.address/name??valuemaster.hadoop:10020/value/propertyproperty??namemapreduce.jobhistory.webapp.addre