sparkonyarn安裝配置手册.docx

ssh无密码登陆安装sshyum install openssh-server产生keyssh-keygen -t rsa -P Enter file in which to save the key (/root/.ssh/id_rsa):（按回车）使用keycat /root/.ssh/id_rsa.pub /root/.ssh/authorized_keys安装配置JDK解压tar -zxvf jdk-7u71-linux-x64.tar.gz打开全局变量配置文件vim /etc/profile在该文件末尾增加如下语句使配置生效source /etc/profile确认JDK安装成功安装配置hadoop解压tar -zxvf hadoop-2.2.0.tar.gz配置 hadoop-env.shcd /opt/hadoop-2.2.0/etc/hadoopvim hadoop-env.sh增加如下配置：在/etc/profile里增加如下配置：尤其最后两行，否则会导致启动错误。配置core-site.xmlcd /opt/hadoop-2.2.0/etc/hadoopvim core-site.xml增加如下配置还需增加如下配置，否则找不到库property namehadoop.native.lib/name valuetrue/value /property配置hdfs-site.xmlcd /opt/hadoop-2.2.0/etc/hadoopvim hdfs-site.xml增加如下配置配置mapred-site.xmlcd /opt/hadoop-2.2.0/etc/hadoopcp mapred-site.xml.template mapred-site.xmlvim mapred-site.xml增加如下配置使配置生效source hadoop-env.sh启动hadoop总是报如下错误WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable原因是apache官网提供的二进制包，里面的native库，是32位的，而服务器是64位的。下载Hadoop 2.2.0 源码包，并解压安装相关软件yum install lzo-devel zlib-devel gcc autoconf automake libtool ncurses-devel openssl-deve安装Maventar zxf apache-maven-3.0.5-bin.tar.gz -C /optvim /etc/profileexport MAVEN_HOME=/opt/apache-maven-3.0.5export PATH=$PATH:$MAVEN_HOME/binsource /etc/profile安装Anttar zxf apache-ant-1.9.3-bin.tar.gz -C /optvim /etc/profileexport ANT_HOME=/opt/apache-ant-1.9.3export PATH=$PATH:$ANT_HOME/binsource /etc/profile安装Findbugstar zxf findbugs-2.0.3.tar.gz -C /optvim /etc/profileexport FINDBUGS_HOME=/opt/findbugs-2.0.3export PATH=$PATH:$FINDBUGS_HOME/binsource /etc/profile安装protobuf$ tar zxf protobuf-2.5.0.tar.gz$ cd protobuf-2.5.0$ ./configure$ make$ sudo make install给hadoop打补丁最新的Hadoop 2.2.0 的Source Code 压缩包解压出来的code有个bug 需要patch后才能编译。否则编译hadoop-auth 会提示错误Patch: /jira/browse/HADOOP-10110下载下来后使用如下拷贝到hadoop目录下，使用如下命令打补丁patch –p0 HADOOP-10110.patch编译hadoopmvn package -DskipTests -Pdist,native -Dtar使用编译好的库替换原来的库rm -rf /opt/hadoop-2.2.0/lib/nativecp ./hadoop-dist/target/hadoop-2.2.0/li