flume包安装手册.docVIP

Flume安装手册 环境要求 linux系统 安装步骤 检查安装环境 首先请检查系统是Linux系统安装 首先将需要安装的文件包apache-flume-1.4.0-bin，或者是该文件的压缩包用解压缩命令解压到指定安装目录。 然后到apache-flume-1.4.0-bin/conf目录下修改配置文件flume-conf.properties，如下说明(主要修改consumer部分)： ############################################ # producer config ########################################### #agent section producer.sources = s producer.channels = c producer.sinks = r #source section producer.sources.s.type = avro producer.sources.s.channels = c producer.sources.s.bind = 10.243.0.250 producer.sources.s.port = 44443 # Each sinks type must be defined producer.sinks.r.type = org.apache.flume.plugins.KafkaSink producer.sinks.r.metadata.broker.list=10.243.0.250:11110 producer.sinks.r.partition.key=0 producer.sinks.r.partitioner.class=org.apache.flume.plugins.SinglePartition producer.sinks.r.serializer.class=kafka.serializer.StringEncoder producer.sinks.r.request.required.acks=0 producer.sinks.r.max.message.size=1000000 producer.sinks.r.producer.type=sync producer.sinks.r.custom.encoding=UTF-8 producer.sinks.r.custom.topic.name=3test producer.sinks.r.hostname = 10.243.0.250 #Specify the channel the sink should use producer.sinks.r.channel = c # Each channels type is defined. producer.channels.c.type = memory producer.channels.c.capacity = 1000 ############################################ # consumer config ########################################### consumer.sources = s consumer.channels = c consumer.sinks = r consumer.sources.s.type = avro //对客户端为java代码时设置为：avro,若为C设置为：syslogudp. consumer.sources.s.bind = 10.243.0.250 //flume的主机IP consumer.sources.s.port = 44443 //提供给用户的端口号 consumer.sources.s.channels = c consumer.sinks.r.channel = c consumer.channels.c.type = memory consumer.channels.c.capacity = 1000000 consumer.sinks.r.type = org.apache.flume.plugins.KafkaSink consumer.sinks.r.metadata.broker.list=10.243.0.250:11110,10.243.0.251:11110,10.243.0.252:11110 //Kafka的IP及端口。 consumer.sinks.r.partition.key=0 consumer.sinks.r.partitioner.class=o