Hive 性能测试.ppt

Hive性能测试;大数据公司Hive的应用情况;测试环境;Hadoop Streaming 测试;hadoop jar \ /usr/lib/hadoop/contrib/streaming/hadoop-streaming-0.20.2-Intel.jar \ -mapper mapper.sh \ -reducer reducer.sh \ -combiner combiner.sh \ -file mapper.sh \ -file reducer.sh \ -file combiner.sh \ -input? /data/input3/ \ -output /user/root/tmp_out66/ \ -jobconf stream.non.zero.exit.is.failure=false 13/09/25 16:45:44 WARN streaming.StreamJob: -jobconf option is deprecated, please use -D instead. packageJobJar: [mapper.sh, reducer.sh, combiner.sh, /tmp/hadoop-root/hadoop-unjar8448429247117545510/] [] /tmp/streamjob6698707843960768352.jar tmpDir=null 13/09/25 16:45:44 WARN snappy.LoadSnappy: Snappy native library is available 13/09/25 16:45:44 INFO util.NativeCodeLoader: Loaded the native-hadoop library 13/09/25 16:45:44 INFO snappy.LoadSnappy: Snappy native library loaded 13/09/25 16:45:44 INFO mapred.FileInputFormat: Total input paths to process : 121 13/09/25 16:45:45 INFO streaming.StreamJob: getLocalDirs(): [/mapred/local] 13/09/25 16:45:45 INFO streaming.StreamJob: Running job: job_201209091642_0039 13/09/25 16:45:45 INFO streaming.StreamJob: To kill this job, run: 13/09/25 16:45:45 INFO streaming.StreamJob: /usr/lib/hadoop-0.20/bin/hadoop job? -Dmapred.job.tracker=node11:54311 -kill job_201209091642_0039 13/09/25 16:45:45 INFO streaming.StreamJob: Tracking URL: http://node11:50030/jobdetails.jsp?jobid=job_201209091642_0039 13/09/25 16:45:46 INFO streaming.StreamJob:? map 0%? reduce 0% 13/09/25 16:45:58 INFO streaming.StreamJob:? map 1%? reduce 0% ….. 13/09/25 16:54:10 INFO streaming.StreamJob:? map 100%? reduce 100% 13/09/25 16:54:11 INFO streaming.StreamJob: Job complete: job_201209091642_0039;Hive 建立外部表;Hive count测试;Hive group测试;Hive 排序测试;Hive join测试;Hive 字符串函数测试;Hive操作Hbase;Hive操作Hbase;Hive的优势;结论;Hive查询的优化1;Hive查询的优化2;Hive查询的优化3;Hive查询的优化4;更大数据集的测试;count(*);count(distinct mobile);更复杂的操作