大数据平台技术实施与运维规范-Hadoop 分册.docx

PAGE \* MERGEFORMAT 1大数据平台技术实施与运维规范Hadoop 分册目 录 TOC \o "1-3" \h \z \u HYPERLINK \l "_Toc508653934" 1 产品概述 PAGEREF _Toc508653934 \h 6 HYPERLINK \l "_Toc508653935" 1.1 组件描述 PAGEREF _Toc508653935 \h 6 HYPERLINK \l "_Toc508653936" 1.1.1 Zookeeper PAGEREF _Toc508653936 \h 6 HYPERLINK \l "_Toc508653937" 1.1.2 HDFS PAGEREF _Toc508653937 \h 7 HYPERLINK \l "_Toc508653938" 1.1.3 Mapreduce PAGEREF _Toc508653938 \h 8 HYPERLINK \l "_Toc508653939" 1.1.4 Yarn PAGEREF _Toc508653939 \h 12 HYPERLINK \l "_Toc508653940" 1.1.5 Hbase PAGEREF _Toc508653940 \h 14 HYPERLINK \l "_Toc508653941" 1.1.6 Hive PAGEREF _Toc508653941 \h 15 HYPERLINK \l "_Toc508653942" 1.1.7 Sqoop PAGEREF _Toc508653942 \h 18 HYPERLINK \l "_Toc508653943" 1.1.8 Spark PAGEREF _Toc508653943 \h 19 HYPERLINK \l "_Toc508653944" 1.1.9 Mahout PAGEREF _Toc508653944 \h 20 HYPERLINK \l "_Toc508653945" 1.1.10 Rhadoop PAGEREF _Toc508653945 \h 20 HYPERLINK \l "_Toc508653946" 2 系统环境 PAGEREF _Toc508653946 \h 20 HYPERLINK \l "_Toc508653947" 2.1 网络配置 PAGEREF _Toc508653947 \h 20 HYPERLINK \l "_Toc508653948" 2.1.1 网卡速率检测 PAGEREF _Toc508653948 \h 20 HYPERLINK \l "_Toc508653949" 2.1.2 网卡绑定方法 PAGEREF _Toc508653949 \h 21 HYPERLINK \l "_Toc508653950" 2.2 创建用户 PAGEREF _Toc508653950 \h 24 HYPERLINK \l "_Toc508653951" 2.3 配置SSH互信 PAGEREF _Toc508653951 \h 24 HYPERLINK \l "_Toc508653952" 2.4 防火墙 PAGEREF _Toc508653952 \h 27 HYPERLINK \l "_Toc508653953" 2.5 Selinux PAGEREF _Toc508653953 \h 28 HYPERLINK \l "_Toc508653954" 2.6 时间同步服务 PAGEREF _Toc508653954 \h 29 HYPERLINK \l "_Toc508653955" 2.7 修改最大进程数和句柄数 PAGEREF _Toc508653955 \h 32 HYPERLINK \l "_Toc508653956" 2.8 修改主机名文件 PAGEREF _Toc508653956 \h 33 HYPERLINK \l "_Toc508653957" 3 安装部署 PAGEREF _Toc508653957 \h 33 HYPERLINK \l "_Toc508653958" 3.1 安装软件包配置环境变量 PAGEREF _Toc508653958 \h 33 HYPERLINK \l "_Toc508653959" 3.1.1 软件包 PAGEREF _Toc508653959 \h 33 HYPERLINK \l "_Toc508653960" 3.1.2 环境变量 PAGEREF _Toc508653960 \h 36 HYPERLINK \l "_Toc508653961" 3.1.3 hadoop相关服务要求 PAGE